asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Blow <mblow.apa...@gmail.com>
Subject Re: External function dependency problem
Date Mon, 04 Jul 2016 06:34:17 GMT
I think a better solution might be to invert the normal class loader
resolution order with an extension of URLClassLoader that overrides
loadClass(), which tries resolving classes internally before delegating to
the parent. Boot classpath classes can still be forced to load correctly. I
think this will solve the use case without the regression due to inability
to resolve *db classes.

Thanks,

-MDB

On Sat, Jul 2, 2016 at 11:44 PM Xikui Wang <xikuiw@uci.edu> wrote:

> The error log is as follow:
>
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at
>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
> at
>
> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
> at
>
> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
> at
>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at
>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
> at
>
> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
> at
>
> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
> at
>
> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
> at
>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
> ... 8 more
> Caused by: java.lang.ClassCastException:
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
> org.apache.asterix.external.api.IFunctionFactory
> at
>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
> ... 12 more
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at
>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
> at
>
> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
> at
>
> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
> at
>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at
>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
> at
>
> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
> at
>
> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
> at
>
> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
> at
>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
> ... 8 more
> Caused by: java.lang.ClassCastException:
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
> org.apache.asterix.external.api.IFunctionFactory
> at
>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
> ... 12 more
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at
>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130)
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
> ... 4 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at
>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
> at
>
> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
> at
>
> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
> at
>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
> ... 5 more
> Caused by:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at
>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
> at
>
> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
> at
>
> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
> at
>
> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
> at
>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
> ... 8 more
> Caused by: java.lang.ClassCastException:
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
> org.apache.asterix.external.api.IFunctionFactory
> at
>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
> ... 12 more
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at
>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130)
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
> ... 4 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at
>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
> at
>
> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
> at
>
> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
> at
>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
> ... 5 more
> Caused by:
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>  Unable to load/instantiate class
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> at
>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
> at
>
> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
> at
>
> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
> at
>
> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
> at
>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
> ... 8 more
> Caused by: java.lang.ClassCastException:
> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
> org.apache.asterix.external.api.IFunctionFactory
> at
>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
> ... 12 more
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> java.lang.InterruptedException
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> java.lang.InterruptedException
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
> ... 4 more
> Caused by: java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
> ... 6 more
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> java.lang.InterruptedException
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> java.lang.InterruptedException
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
> ... 4 more
> Caused by: java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
> ... 6 more
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> java.lang.InterruptedException
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> java.lang.InterruptedException
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
> ... 4 more
> Caused by: java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
> ... 6 more
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> java.lang.InterruptedException
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> java.lang.InterruptedException
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
> ... 4 more
> Caused by: java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at
>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
> ... 6 more
> testFile
>
> src/test/resources/runtimets/queries/external-library/tweetGeoTag/tweetGeoTag.3.update.aql
> raised an exception:
> java.lang.Exception: HTTP operation failed: 2
> STATUS LINE: HTTP/1.1 500 Server Error
> SUMMARY: Error: Failure in feed
>
> STACKTRACE: org.apache.asterix.common.exceptions.AsterixException: Failure
> in feed
> at
>
> org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertNoFailure(FeedLifecycleEventSubscriber.java:62)
> at
>
> org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertEvent(FeedLifecycleEventSubscriber.java:55)
> at
>
> org.apache.asterix.aql.translator.QueryTranslator.handleConnectFeedStatement(QueryTranslator.java:2238)
> at
>
> org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:366)
> at
>
> org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:253)
> at
>
> org.apache.asterix.api.http.servlet.RESTAPIServlet.handleRequest(RESTAPIServlet.java:195)
> at
>
> org.apache.asterix.api.http.servlet.UpdateAPIServlet.handleRequest(UpdateAPIServlet.java:30)
> at
>
> org.apache.asterix.api.http.servlet.RESTAPIServlet.doPost(RESTAPIServlet.java:162)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
> at
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:228)
> at
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:956)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
> at
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:188)
> at
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:891)
> at
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> at
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114)
> at org.eclipse.jetty.server.Server.handle(Server.java:353)
> at
>
> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:598)
> at
>
> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1076)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:805)
> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:218)
> at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:427)
> at
>
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:510)
> at
>
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34)
> at
>
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
> at
>
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:450)
> at java.lang.Thread.run(Thread.java:745)
>
> at
>
> org.apache.asterix.test.aql.TestExecutor.executeHttpMethod(TestExecutor.java:275)
> at
>
> org.apache.asterix.test.aql.TestExecutor.executeUpdate(TestExecutor.java:331)
> at
> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:496)
> at
> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:762)
> at
> org.apache.asterix.test.runtime.ExecutionTest.test(ExecutionTest.java:106)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
>
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
>
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
>
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
>
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
>
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> at
>
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119)
> at
>
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
> at
>
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
> ...Unexpected!
>
> On Sat, Jul 2, 2016 at 7:23 PM, Raman Grover <ramangrover29@gmail.com>
> wrote:
>
> > i am missing the attachment
> > On Jul 2, 2016 6:39 PM, "Xikui Wang" <xikuiw@uci.edu> wrote:
> >
> > > Hi Raman,
> > >
> > > Thanks for your help. I tried this quick fix on my branch, but it
> > > introduces some new exceptions. I think this causes Asterix fails at
> > > entering the external function. The error message is attached.
> > >
> > > Best,
> > > Xikui
> > >
> > > On Fri, Jul 1, 2016 at 10:11 AM, Raman Grover <ramangrover29@gmail.com
> >
> > > wrote:
> > >
> > >> Operations related to setting up an external library are contained in
> > >> ExternalLibraryUtil
> > >> <
> > >>
> >
> https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
> > >> >
> > >>
> > >> At line 382, we have
> > >> // create and return the class loader
> > >>
> > >> ClassLoader classLoader = new URLClassLoader(urls, parentClassLoader);
> > >> return classLoader;
> > >>
> > >> Above, we have the parentClassLoader set to the classloader for
> > >> ExternalLibraryUtil which is the application class loader (AsterixDB's
> > >> classloader that loads the dependencies from pom.xml). The proposed
> > >> solution (a) in earlier thread - skipping application classloader
> would
> > >> translate to replacing the above code with
> > >>
> > >> ClassLoader classLoader = new URLClassLoader(urls, null);
> > >>
> > >> Regards,
> > >> Raman
> > >>
> > >> On Thu, Jun 30, 2016 at 4:41 PM, Xikui Wang <xikuiw@uci.edu> wrote:
> > >>
> > >> > >
> > >> > > Hi Abdullah,
> > >> >
> > >> > I reverted my code to reproduce the problem. Noticing this external
> > >> > function has a couple of other bugs but the dependency one is
> blocking
> > >> > others, so this should be enough to reproduce the problem.
> > >> >
> > >> > >
> > >> > The external function package is geoTag.zip.
> > >> >
> > >> > > ‚Äč
> > >> > >
> > >> > > ‚ÄčTest scripts are in tweetGeoTag.zip
> > >> >
> > >> > External function is loading data from data/, i.e.: data/state.json
> .
> > So
> > >> > all json files in data.zip need to be placed under
> > >> > ../asterixdb/asterix-app/data/
> > >> >
> > >> > The real_tweets_adm.adm used in ddl is also attached.
> > >> >
> > >> > This setting will cause
> > >> >
> > >> > java.lang.NoSuchMethodError:
> > >> > > com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
> > >> > > at
> > >> > >
> > >>
> > com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:541)
> > >> > > at
> > >> > >
> > >>
> > com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:452)
> > >> > > at
> > org.wololo.geojson.GeoJSONFactory.<clinit>(GeoJSONFactory.java:17)
> > >> > > at
> > >> > >
> > >> >
> > >>
> >
> edu.uci.ics.cloudberry.gnosis.USGeoJSONIndex.loadShape(IGeoIndex.scala:29)
> > >> > > at
> > >> > >
> > >> >
> > >>
> >
> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$.loadShape(USGeoGnosis.scala:101)
> > >> > > at
> > >> > >
> > >> >
> > >>
> >
> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:20)
> > >> > > at
> > >> > >
> > >> >
> > >>
> >
> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:18)
> > >> > > at
> > >> > >
> > >> >
> > >>
> >
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
> > >> > > at
> > >> > >
> > >> >
> > >>
> >
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
> > >> > > at scala.collection.immutable.List.foreach(List.scala:381)
> > >> > > at
> > >> scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
> > >> > > at scala.collection.immutable.List.map(List.scala:285)
> > >> > > at
> > >> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.load(USGeoGnosis.scala:18)
> > >> > > at
> > >> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.<init>(USGeoGnosis.scala:15)
> > >> > > at
> > >> > >
> > >> >
> > >>
> >
> edu.uci.ics.cloudberry.noah.feed.GeoTagFunction.initialize(GeoTagFunction.java:34)
> > >> > > at
> > >> > >
> > >> >
> > >>
> >
> org.apache.asterix.external.library.ExternalFunction.initialize(ExternalFunction.java:113)
> > >> > >
> > >> >
> > >> > The attachments exceed the size limits. I moved them under this
> google
> > >> > drive directory:
> > >> >
> > >> >
> > >> >
> > >>
> >
> https://drive.google.com/folderview?id=0B_6Dzy3OTjaNRUVJWTRvWEtKSU0&usp=sharing
> > >> >
> > >> > Let me know if you need anything else. Thanks.
> > >> >
> > >> > Best,
> > >> > Xikui
> > >> >
> > >> > On Thu, Jun 30, 2016 at 12:06 PM, abdullah alamoudi <
> > bamousaa@gmail.com
> > >> >
> > >> > wrote:
> > >> >
> > >> > > I will look into that.
> > >> > >>
> > >> > >> @Xikui,
> > >> > >> Do you have a reproducable use case where such a behavior
can be
> > >> > observed?
> > >> > >>
> > >> > >> Cheers,
> > >> > >> Abdullah.
> > >> > >>
> > >> > >> On Thu, Jun 30, 2016 at 7:53 PM, Raman Grover <
> > >> ramangrover29@gmail.com>
> > >> > >> wrote:
> > >> > >>
> > >> > >> > Hi
> > >> > >> >
> > >> > >> > Each external library has an separately associated custom
class
> > >> loader
> > >> > >> > (URLClassLoader) that loads the contained functions
and any
> > >> > dependencies
> > >> > >> > (jar files) packaged inside the library. The custom
loader fits
> > >> into
> > >> > the
> > >> > >> > natural hierarchy with system (classpath based) classloader
as
> > the
> > >> > >> > immediate parent and bootstrap classloader as the root.
As the
> > >> > >> delegation
> > >> > >> > model works, a class loader attempts to load a class
only after
> > >> > >> parent(s)
> > >> > >> > in the lineage have failed.
> > >> > >> >
> > >> > >> > In your specific case, the system class loader is able
to find
> > the
> > >> > >> jackson
> > >> > >> > library from the configured classpath and so the library
> > >> classloader
> > >> > is
> > >> > >> not
> > >> > >> > involved at all for this jar. The problem arising due
to
> > different
> > >> > >> versions
> > >> > >> > of a jar present across the hierarchy of classloaders
is
> similar
> > to
> > >> > one
> > >> > >> > faced in web containers such as Tomcat or application
servers
> > such
> > >> as
> > >> > >> > Websphere where system loaded classes can override any
> > application
> > >> > >> specific
> > >> > >> > classes due to name conflict.
> > >> > >> >
> > >> > >> > The way you are defining a library is correct. The fix
needs to
> > be
> > >> in
> > >> > >> the
> > >> > >> > way the classes are loaded by AsterixDB (here acting
as an
> > >> application
> > >> > >> > server).
> > >> > >> >
> > >> > >> > I see two ways:-
> > >> > >> >
> > >> > >> > a) Skipping the system class loader: AsterixDB uses
a
> > >> URLClassloader
> > >> > to
> > >> > >> > load libraries. Setting the  parent of the this classloader
as
> > null
> > >> > >> implies
> > >> > >> > the bootrstrap classloader (root) as the immediate parent.
The
> > >> > bootstrap
> > >> > >> > classloader loads rt.jar (java runtime classes) and
i18n.jar
> and
> > >> these
> > >> > >> > definitely do not have conflicts with your library dependencies
> > (if
> > >> > they
> > >> > >> > do, not sure why one would override the classes in rt.jar
and
> so
> > we
> > >> > >> need to
> > >> > >> > fix the library). Note that the system classloader is
out of
> the
> > >> > >> hierarchy
> > >> > >> > here and so any other jars loaded by AsterixDB from
its
> classpath
> > >> > >> (pom.xml)
> > >> > >> > would not be visible.
> > >> > >> >
> > >> > >> > b) Defining a Custom Class loader: Inside AsterixDB,
we
> provide a
> > >> > custom
> > >> > >> > implementation of classloader that attempts to load
a class
> prior
> > >> to
> > >> > >> > delegating to the parent. This way, the library classes
and
> > >> packaged
> > >> > >> > dependencies override any system level classes from
the class
> > path
> > >> and
> > >> > >> even
> > >> > >> > the classes contained in rt.jar and i18n.jar.
> > >> > >> >
> > >> > >> > I am opening the discussion here to suggest further
> alternatives
> > or
> > >> > >> provide
> > >> > >> > preferences.
> > >> > >> >
> > >> > >> > I have a preference for (a)  (skipping the system class
loader)
> > for
> > >> > two
> > >> > >> > reasons:
> > >> > >> >
> > >> > >> > a) it is simpler
> > >> > >> >
> > >> > >> > b) the other option allows a custom class loader to
override
> > >> classes
> > >> > in
> > >> > >> > rt.jar., which is ok but not how classloaders are supposed
to
> > work
> > >> in
> > >> > >> > principle.
> > >> > >> >
> > >> > >> > Regards,
> > >> > >> >
> > >> > >> > Raman
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > On Jun 29, 2016 11:07 PM, "Mike Carey" <dtabass@gmail.com>
> > wrote:
> > >> > >> >
> > >> > >> > > Any classloader experts have suggestions...?
> > >> > >> > > On Jun 29, 2016 10:26 PM, "Xikui Wang" <xikuiw@uci.edu>
> wrote:
> > >> > >> > >
> > >> > >> > > > Hi Devs,
> > >> > >> > > >
> > >> > >> > > > We found a problem when trying to build external
functions
> > for
> > >> the
> > >> > >> > > > cloudberry demo.
> > >> > >> > > >
> > >> > >> > > > When the external function is depend on certain
library,
> the
> > >> > library
> > >> > >> > that
> > >> > >> > > > comes with the external function will be blocked
by same
> > >> library
> > >> > in
> > >> > >> > > > AsterixDB. In our case, our external function
'geoTag' uses
> > >> > jackson
> > >> > >> > > v2.7.1,
> > >> > >> > > > and we packed all dependencies into one single
jar. When
> > >> running
> > >> > >> > 'geoTag'
> > >> > >> > > > on Asterix, it will call jackson v2.0.0 in
AsterixDB which
> > >> causes
> > >> > >> > > > NullPointerException. We have to manually
change pom.xml in
> > >> > >> AsterixDB
> > >> > >> > to
> > >> > >> > > > fix that.
> > >> > >> > > >
> > >> > >> > > > We are wondering is that because we load the
external
> > function
> > >> in
> > >> > a
> > >> > >> > wrong
> > >> > >> > > > way, or this could be one possible interesting
problem
> which
> > is
> > >> > >> worth
> > >> > >> > > > noticing. Thanks.
> > >> > >> > > >
> > >> > >> > > > Best,
> > >> > >> > > > Xikui
> > >> > >> > > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> > >
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Raman
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message