asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Blow <mblow.apa...@gmail.com>
Subject Re: External function dependency problem
Date Mon, 04 Jul 2016 07:36:38 GMT
I've submitted a patchset to Gerrit with a proposed fix for this:

https://asterix-gerrit.ics.uci.edu/#/c/973/

Thanks,

-MDB
On Mon, Jul 4, 2016 at 2:37 AM Till Westmann <tillw@apache.org> wrote:

> Sounds good to me!
>
> Cheers,
> Till
>
> On 3 Jul 2016, at 23:34, Michael Blow wrote:
>
> > I think a better solution might be to invert the normal class loader
> > resolution order with an extension of URLClassLoader that overrides
> > loadClass(), which tries resolving classes internally before
> > delegating to
> > the parent. Boot classpath classes can still be forced to load
> > correctly. I
> > think this will solve the use case without the regression due to
> > inability
> > to resolve *db classes.
> >
> > Thanks,
> >
> > -MDB
> >
> > On Sat, Jul 2, 2016 at 11:44 PM Xikui Wang <xikuiw@uci.edu> wrote:
> >
> >> The error log is as follow:
> >>
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
> >> at
> >>
> >>
> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
> >> at
> >>
> >>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
> >> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
> >> ... 8 more
> >> Caused by: java.lang.ClassCastException:
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
> >> org.apache.asterix.external.api.IFunctionFactory
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
> >> ... 12 more
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
> >> at
> >>
> >>
> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
> >> at
> >>
> >>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
> >> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
> >> ... 8 more
> >> Caused by: java.lang.ClassCastException:
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
> >> org.apache.asterix.external.api.IFunctionFactory
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
> >> ... 12 more
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> >> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at
> >>
> >>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130)
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
> >> ... 4 more
> >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
> >> at
> >>
> >>
> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
> >> at
> >>
> >>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
> >> ... 5 more
> >> Caused by:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
> >> ... 8 more
> >> Caused by: java.lang.ClassCastException:
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
> >> org.apache.asterix.external.api.IFunctionFactory
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
> >> ... 12 more
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> >> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at
> >>
> >>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130)
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
> >> ... 4 more
> >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
> >> at
> >>
> >>
> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
> >> at
> >>
> >>
> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
> >> ... 5 more
> >> Caused by:
> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> >>  Unable to load/instantiate class
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
> >> at
> >>
> >>
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
> >> ... 8 more
> >> Caused by: java.lang.ClassCastException:
> >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
> >> org.apache.asterix.external.api.IFunctionFactory
> >> at
> >>
> >>
> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
> >> ... 12 more
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> java.lang.InterruptedException
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> >> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> >> java.lang.InterruptedException
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
> >> ... 4 more
> >> Caused by: java.lang.InterruptedException
> >> at java.lang.Object.wait(Native Method)
> >> at java.lang.Object.wait(Object.java:502)
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
> >> ... 6 more
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> java.lang.InterruptedException
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> >> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> >> java.lang.InterruptedException
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
> >> ... 4 more
> >> Caused by: java.lang.InterruptedException
> >> at java.lang.Object.wait(Native Method)
> >> at java.lang.Object.wait(Object.java:502)
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
> >> ... 6 more
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> java.lang.InterruptedException
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> >> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> >> java.lang.InterruptedException
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
> >> ... 4 more
> >> Caused by: java.lang.InterruptedException
> >> at java.lang.Object.wait(Native Method)
> >> at java.lang.Object.wait(Object.java:502)
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
> >> ... 6 more
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> org.apache.hyracks.api.exceptions.HyracksDataException:
> >> java.lang.InterruptedException
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> >> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> >> java.lang.InterruptedException
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
> >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
> >> ... 4 more
> >> Caused by: java.lang.InterruptedException
> >> at java.lang.Object.wait(Native Method)
> >> at java.lang.Object.wait(Object.java:502)
> >> at
> >>
> >>
> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
> >> ... 6 more
> >> testFile
> >>
> >>
> src/test/resources/runtimets/queries/external-library/tweetGeoTag/tweetGeoTag.3.update.aql
> >> raised an exception:
> >> java.lang.Exception: HTTP operation failed: 2
> >> STATUS LINE: HTTP/1.1 500 Server Error
> >> SUMMARY: Error: Failure in feed
> >>
> >> STACKTRACE: org.apache.asterix.common.exceptions.AsterixException:
> >> Failure
> >> in feed
> >> at
> >>
> >>
> org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertNoFailure(FeedLifecycleEventSubscriber.java:62)
> >> at
> >>
> >>
> org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertEvent(FeedLifecycleEventSubscriber.java:55)
> >> at
> >>
> >>
> org.apache.asterix.aql.translator.QueryTranslator.handleConnectFeedStatement(QueryTranslator.java:2238)
> >> at
> >>
> >>
> org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:366)
> >> at
> >>
> >>
> org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:253)
> >> at
> >>
> >>
> org.apache.asterix.api.http.servlet.RESTAPIServlet.handleRequest(RESTAPIServlet.java:195)
> >> at
> >>
> >>
> org.apache.asterix.api.http.servlet.UpdateAPIServlet.handleRequest(UpdateAPIServlet.java:30)
> >> at
> >>
> >>
> org.apache.asterix.api.http.servlet.RESTAPIServlet.doPost(RESTAPIServlet.java:162)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> >> at
> >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
> >> at
> >>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
> >> at
> >>
> >>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:228)
> >> at
> >>
> >>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:956)
> >> at
> >>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
> >> at
> >>
> >>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:188)
> >> at
> >>
> >>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:891)
> >> at
> >>
> >>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> >> at
> >>
> >>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114)
> >> at org.eclipse.jetty.server.Server.handle(Server.java:353)
> >> at
> >>
> >>
> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:598)
> >> at
> >>
> >>
> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1076)
> >> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:805)
> >> at
> >> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:218)
> >> at
> >> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:427)
> >> at
> >>
> >>
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:510)
> >> at
> >>
> >>
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34)
> >> at
> >>
> >>
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
> >> at
> >>
> >>
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:450)
> >> at java.lang.Thread.run(Thread.java:745)
> >>
> >> at
> >>
> >>
> org.apache.asterix.test.aql.TestExecutor.executeHttpMethod(TestExecutor.java:275)
> >> at
> >>
> >>
> org.apache.asterix.test.aql.TestExecutor.executeUpdate(TestExecutor.java:331)
> >> at
> >>
> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:496)
> >> at
> >>
> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:762)
> >> at
> >>
> org.apache.asterix.test.runtime.ExecutionTest.test(ExecutionTest.java:106)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >> at
> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:497)
> >> at
> >>
> >>
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> >> at
> >>
> >>
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> >> at
> >>
> >>
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> >> at
> >>
> >>
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> >> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> >> at
> >>
> >>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> >> at
> >>
> >>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> >> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> >> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> >> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> >> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> >> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> >> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> >> at org.junit.runners.Suite.runChild(Suite.java:127)
> >> at org.junit.runners.Suite.runChild(Suite.java:26)
> >> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> >> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> >> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> >> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> >> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> >> at
> >>
> >>
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> >> at
> >>
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> >> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> >> at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> >> at
> >>
> >>
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119)
> >> at
> >>
> >>
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
> >> at
> >>
> >>
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> >> at
> >> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >> at
> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:497)
> >> at
> >> com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
> >> ...Unexpected!
> >>
> >> On Sat, Jul 2, 2016 at 7:23 PM, Raman Grover
> >> <ramangrover29@gmail.com>
> >> wrote:
> >>
> >>> i am missing the attachment
> >>> On Jul 2, 2016 6:39 PM, "Xikui Wang" <xikuiw@uci.edu> wrote:
> >>>
> >>>> Hi Raman,
> >>>>
> >>>> Thanks for your help. I tried this quick fix on my branch, but it
> >>>> introduces some new exceptions. I think this causes Asterix fails
> >>>> at
> >>>> entering the external function. The error message is attached.
> >>>>
> >>>> Best,
> >>>> Xikui
> >>>>
> >>>> On Fri, Jul 1, 2016 at 10:11 AM, Raman Grover
> >>>> <ramangrover29@gmail.com
> >>>
> >>>> wrote:
> >>>>
> >>>>> Operations related to setting up an external library are contained
> >>>>> in
> >>>>> ExternalLibraryUtil
> >>>>> <
> >>>>>
> >>>
> >>
> https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
> >>>>>>
> >>>>>
> >>>>> At line 382, we have
> >>>>> // create and return the class loader
> >>>>>
> >>>>> ClassLoader classLoader = new URLClassLoader(urls,
> >>>>> parentClassLoader);
> >>>>> return classLoader;
> >>>>>
> >>>>> Above, we have the parentClassLoader set to the classloader for
> >>>>> ExternalLibraryUtil which is the application class loader
> >>>>> (AsterixDB's
> >>>>> classloader that loads the dependencies from pom.xml). The
> >>>>> proposed
> >>>>> solution (a) in earlier thread - skipping application classloader
> >> would
> >>>>> translate to replacing the above code with
> >>>>>
> >>>>> ClassLoader classLoader = new URLClassLoader(urls, null);
> >>>>>
> >>>>> Regards,
> >>>>> Raman
> >>>>>
> >>>>> On Thu, Jun 30, 2016 at 4:41 PM, Xikui Wang <xikuiw@uci.edu>
> >>>>> wrote:
> >>>>>
> >>>>>>>
> >>>>>>> Hi Abdullah,
> >>>>>>
> >>>>>> I reverted my code to reproduce the problem. Noticing this
> >>>>>> external
> >>>>>> function has a couple of other bugs but the dependency one is
> >> blocking
> >>>>>> others, so this should be enough to reproduce the problem.
> >>>>>>
> >>>>>>>
> >>>>>> The external function package is geoTag.zip.
> >>>>>>
> >>>>>>> ‚Äč
> >>>>>>>
> >>>>>>> ‚ÄčTest scripts are in tweetGeoTag.zip
> >>>>>>
> >>>>>> External function is loading data from data/, i.e.:
> >>>>>> data/state.json
> >> .
> >>> So
> >>>>>> all json files in data.zip need to be placed under
> >>>>>> ../asterixdb/asterix-app/data/
> >>>>>>
> >>>>>> The real_tweets_adm.adm used in ddl is also attached.
> >>>>>>
> >>>>>> This setting will cause
> >>>>>>
> >>>>>> java.lang.NoSuchMethodError:
> >>>>>>> com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
> >>>>>>> at
> >>>>>>>
> >>>>>
> >>>
> com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:541)
> >>>>>>> at
> >>>>>>>
> >>>>>
> >>>
> com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:452)
> >>>>>>> at
> >>> org.wololo.geojson.GeoJSONFactory.<clinit>(GeoJSONFactory.java:17)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> edu.uci.ics.cloudberry.gnosis.USGeoJSONIndex.loadShape(IGeoIndex.scala:29)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$.loadShape(USGeoGnosis.scala:101)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:20)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:18)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
> >>>>>>> at scala.collection.immutable.List.foreach(List.scala:381)
> >>>>>>> at
> >>>>> scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
> >>>>>>> at scala.collection.immutable.List.map(List.scala:285)
> >>>>>>> at
> >>>>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.load(USGeoGnosis.scala:18)
> >>>>>>> at
> >>>>>
> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.<init>(USGeoGnosis.scala:15)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> edu.uci.ics.cloudberry.noah.feed.GeoTagFunction.initialize(GeoTagFunction.java:34)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> org.apache.asterix.external.library.ExternalFunction.initialize(ExternalFunction.java:113)
> >>>>>>>
> >>>>>>
> >>>>>> The attachments exceed the size limits. I moved them under this
> >> google
> >>>>>> drive directory:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> https://drive.google.com/folderview?id=0B_6Dzy3OTjaNRUVJWTRvWEtKSU0&usp=sharing
> >>>>>>
> >>>>>> Let me know if you need anything else. Thanks.
> >>>>>>
> >>>>>> Best,
> >>>>>> Xikui
> >>>>>>
> >>>>>> On Thu, Jun 30, 2016 at 12:06 PM, abdullah alamoudi <
> >>> bamousaa@gmail.com
> >>>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I will look into that.
> >>>>>>>>
> >>>>>>>> @Xikui,
> >>>>>>>> Do you have a reproducable use case where such a behavior
can
> >>>>>>>> be
> >>>>>> observed?
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Abdullah.
> >>>>>>>>
> >>>>>>>> On Thu, Jun 30, 2016 at 7:53 PM, Raman Grover <
> >>>>> ramangrover29@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi
> >>>>>>>>>
> >>>>>>>>> Each external library has an separately associated
custom
> >>>>>>>>> class
> >>>>> loader
> >>>>>>>>> (URLClassLoader) that loads the contained functions
and any
> >>>>>> dependencies
> >>>>>>>>> (jar files) packaged inside the library. The custom
loader
> >>>>>>>>> fits
> >>>>> into
> >>>>>> the
> >>>>>>>>> natural hierarchy with system (classpath based)
classloader as
> >>> the
> >>>>>>>>> immediate parent and bootstrap classloader as the
root. As the
> >>>>>>>> delegation
> >>>>>>>>> model works, a class loader attempts to load a class
only
> >>>>>>>>> after
> >>>>>>>> parent(s)
> >>>>>>>>> in the lineage have failed.
> >>>>>>>>>
> >>>>>>>>> In your specific case, the system class loader is
able to find
> >>> the
> >>>>>>>> jackson
> >>>>>>>>> library from the configured classpath and so the
library
> >>>>> classloader
> >>>>>> is
> >>>>>>>> not
> >>>>>>>>> involved at all for this jar. The problem arising
due to
> >>> different
> >>>>>>>> versions
> >>>>>>>>> of a jar present across the hierarchy of classloaders
is
> >> similar
> >>> to
> >>>>>> one
> >>>>>>>>> faced in web containers such as Tomcat or application
servers
> >>> such
> >>>>> as
> >>>>>>>>> Websphere where system loaded classes can override
any
> >>> application
> >>>>>>>> specific
> >>>>>>>>> classes due to name conflict.
> >>>>>>>>>
> >>>>>>>>> The way you are defining a library is correct. The
fix needs
> >>>>>>>>> to
> >>> be
> >>>>> in
> >>>>>>>> the
> >>>>>>>>> way the classes are loaded by AsterixDB (here acting
as an
> >>>>> application
> >>>>>>>>> server).
> >>>>>>>>>
> >>>>>>>>> I see two ways:-
> >>>>>>>>>
> >>>>>>>>> a) Skipping the system class loader: AsterixDB uses
a
> >>>>> URLClassloader
> >>>>>> to
> >>>>>>>>> load libraries. Setting the  parent of the this
classloader as
> >>> null
> >>>>>>>> implies
> >>>>>>>>> the bootrstrap classloader (root) as the immediate
parent. The
> >>>>>> bootstrap
> >>>>>>>>> classloader loads rt.jar (java runtime classes)
and i18n.jar
> >> and
> >>>>> these
> >>>>>>>>> definitely do not have conflicts with your library
> >>>>>>>>> dependencies
> >>> (if
> >>>>>> they
> >>>>>>>>> do, not sure why one would override the classes
in rt.jar and
> >> so
> >>> we
> >>>>>>>> need to
> >>>>>>>>> fix the library). Note that the system classloader
is out of
> >> the
> >>>>>>>> hierarchy
> >>>>>>>>> here and so any other jars loaded by AsterixDB from
its
> >> classpath
> >>>>>>>> (pom.xml)
> >>>>>>>>> would not be visible.
> >>>>>>>>>
> >>>>>>>>> b) Defining a Custom Class loader: Inside AsterixDB,
we
> >> provide a
> >>>>>> custom
> >>>>>>>>> implementation of classloader that attempts to load
a class
> >> prior
> >>>>> to
> >>>>>>>>> delegating to the parent. This way, the library
classes and
> >>>>> packaged
> >>>>>>>>> dependencies override any system level classes from
the class
> >>> path
> >>>>> and
> >>>>>>>> even
> >>>>>>>>> the classes contained in rt.jar and i18n.jar.
> >>>>>>>>>
> >>>>>>>>> I am opening the discussion here to suggest further
> >> alternatives
> >>> or
> >>>>>>>> provide
> >>>>>>>>> preferences.
> >>>>>>>>>
> >>>>>>>>> I have a preference for (a)  (skipping the system
class
> >>>>>>>>> loader)
> >>> for
> >>>>>> two
> >>>>>>>>> reasons:
> >>>>>>>>>
> >>>>>>>>> a) it is simpler
> >>>>>>>>>
> >>>>>>>>> b) the other option allows a custom class loader
to override
> >>>>> classes
> >>>>>> in
> >>>>>>>>> rt.jar., which is ok but not how classloaders are
supposed to
> >>> work
> >>>>> in
> >>>>>>>>> principle.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>>
> >>>>>>>>> Raman
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Jun 29, 2016 11:07 PM, "Mike Carey" <dtabass@gmail.com>
> >>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Any classloader experts have suggestions...?
> >>>>>>>>>> On Jun 29, 2016 10:26 PM, "Xikui Wang" <xikuiw@uci.edu>
> >> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Devs,
> >>>>>>>>>>>
> >>>>>>>>>>> We found a problem when trying to build
external functions
> >>> for
> >>>>> the
> >>>>>>>>>>> cloudberry demo.
> >>>>>>>>>>>
> >>>>>>>>>>> When the external function is depend on
certain library,
> >> the
> >>>>>> library
> >>>>>>>>> that
> >>>>>>>>>>> comes with the external function will be
blocked by same
> >>>>> library
> >>>>>> in
> >>>>>>>>>>> AsterixDB. In our case, our external function
'geoTag' uses
> >>>>>> jackson
> >>>>>>>>>> v2.7.1,
> >>>>>>>>>>> and we packed all dependencies into one
single jar. When
> >>>>> running
> >>>>>>>>> 'geoTag'
> >>>>>>>>>>> on Asterix, it will call jackson v2.0.0
in AsterixDB which
> >>>>> causes
> >>>>>>>>>>> NullPointerException. We have to manually
change pom.xml in
> >>>>>>>> AsterixDB
> >>>>>>>>> to
> >>>>>>>>>>> fix that.
> >>>>>>>>>>>
> >>>>>>>>>>> We are wondering is that because we load
the external
> >>> function
> >>>>> in
> >>>>>> a
> >>>>>>>>> wrong
> >>>>>>>>>>> way, or this could be one possible interesting
problem
> >> which
> >>> is
> >>>>>>>> worth
> >>>>>>>>>>> noticing. Thanks.
> >>>>>>>>>>>
> >>>>>>>>>>> Best,
> >>>>>>>>>>> Xikui
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Raman
> >>>>>
> >>>>
> >>>>
> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message