asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Westmann" <ti...@apache.org>
Subject Re: External function dependency problem
Date Tue, 05 Jul 2016 03:26:07 GMT
The change looks good to me and I’ve +2’ed it. However, I haven’t
tested it myself. It would be good if Xikui could test it (either
before of after merging to master) and if we could add a test case
that tests the intended behavior.

Cheers,
Till

On 4 Jul 2016, at 0:36, Michael Blow wrote:

> I've submitted a patchset to Gerrit with a proposed fix for this:
>
> https://asterix-gerrit.ics.uci.edu/#/c/973/
>
> Thanks,
>
> -MDB
> On Mon, Jul 4, 2016 at 2:37 AM Till Westmann <tillw@apache.org> wrote:
>
>> Sounds good to me!
>>
>> Cheers,
>> Till
>>
>> On 3 Jul 2016, at 23:34, Michael Blow wrote:
>>
>>> I think a better solution might be to invert the normal class loader
>>> resolution order with an extension of URLClassLoader that overrides
>>> loadClass(), which tries resolving classes internally before
>>> delegating to
>>> the parent. Boot classpath classes can still be forced to load
>>> correctly. I
>>> think this will solve the use case without the regression due to
>>> inability
>>> to resolve *db classes.
>>>
>>> Thanks,
>>>
>>> -MDB
>>>
>>> On Sat, Jul 2, 2016 at 11:44 PM Xikui Wang <xikuiw@uci.edu> wrote:
>>>
>>>> The error log is as follow:
>>>>
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
>>>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
>>>> ... 8 more
>>>> Caused by: java.lang.ClassCastException:
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
>>>> org.apache.asterix.external.api.IFunctionFactory
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
>>>> ... 12 more
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
>>>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
>>>> ... 8 more
>>>> Caused by: java.lang.ClassCastException:
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
>>>> org.apache.asterix.external.api.IFunctionFactory
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
>>>> ... 12 more
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>>>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130)
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
>>>> ... 4 more
>>>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
>>>> ... 5 more
>>>> Caused by:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
>>>> ... 8 more
>>>> Caused by: java.lang.ClassCastException:
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
>>>> org.apache.asterix.external.api.IFunctionFactory
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
>>>> ... 12 more
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>>>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130)
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
>>>> ... 4 more
>>>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
>>>> ... 5 more
>>>> Caused by:
>>>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>>>  Unable to load/instantiate class
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
>>>> ... 8 more
>>>> Caused by: java.lang.ClassCastException:
>>>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
>>>> org.apache.asterix.external.api.IFunctionFactory
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
>>>> ... 12 more
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> java.lang.InterruptedException
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>>>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> java.lang.InterruptedException
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
>>>> ... 4 more
>>>> Caused by: java.lang.InterruptedException
>>>> at java.lang.Object.wait(Native Method)
>>>> at java.lang.Object.wait(Object.java:502)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
>>>> ... 6 more
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> java.lang.InterruptedException
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>>>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> java.lang.InterruptedException
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
>>>> ... 4 more
>>>> Caused by: java.lang.InterruptedException
>>>> at java.lang.Object.wait(Native Method)
>>>> at java.lang.Object.wait(Object.java:502)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
>>>> ... 6 more
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> java.lang.InterruptedException
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>>>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> java.lang.InterruptedException
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
>>>> ... 4 more
>>>> Caused by: java.lang.InterruptedException
>>>> at java.lang.Object.wait(Native Method)
>>>> at java.lang.Object.wait(Object.java:502)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
>>>> ... 6 more
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> java.lang.InterruptedException
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>>>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>>>> java.lang.InterruptedException
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
>>>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
>>>> ... 4 more
>>>> Caused by: java.lang.InterruptedException
>>>> at java.lang.Object.wait(Native Method)
>>>> at java.lang.Object.wait(Object.java:502)
>>>> at
>>>>
>>>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
>>>> ... 6 more
>>>> testFile
>>>>
>>>>
>> src/test/resources/runtimets/queries/external-library/tweetGeoTag/tweetGeoTag.3.update.aql
>>>> raised an exception:
>>>> java.lang.Exception: HTTP operation failed: 2
>>>> STATUS LINE: HTTP/1.1 500 Server Error
>>>> SUMMARY: Error: Failure in feed
>>>>
>>>> STACKTRACE: org.apache.asterix.common.exceptions.AsterixException:
>>>> Failure
>>>> in feed
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertNoFailure(FeedLifecycleEventSubscriber.java:62)
>>>> at
>>>>
>>>>
>> org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertEvent(FeedLifecycleEventSubscriber.java:55)
>>>> at
>>>>
>>>>
>> org.apache.asterix.aql.translator.QueryTranslator.handleConnectFeedStatement(QueryTranslator.java:2238)
>>>> at
>>>>
>>>>
>> org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:366)
>>>> at
>>>>
>>>>
>> org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:253)
>>>> at
>>>>
>>>>
>> org.apache.asterix.api.http.servlet.RESTAPIServlet.handleRequest(RESTAPIServlet.java:195)
>>>> at
>>>>
>>>>
>> org.apache.asterix.api.http.servlet.UpdateAPIServlet.handleRequest(UpdateAPIServlet.java:30)
>>>> at
>>>>
>>>>
>> org.apache.asterix.api.http.servlet.RESTAPIServlet.doPost(RESTAPIServlet.java:162)
>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>>> at
>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
>>>> at
>>>>
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:228)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:956)
>>>> at
>>>>
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:188)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:891)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114)
>>>> at org.eclipse.jetty.server.Server.handle(Server.java:353)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:598)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1076)
>>>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:805)
>>>> at
>>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:218)
>>>> at
>>>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:427)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:510)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
>>>> at
>>>>
>>>>
>> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:450)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>>
>>>> at
>>>>
>>>>
>> org.apache.asterix.test.aql.TestExecutor.executeHttpMethod(TestExecutor.java:275)
>>>> at
>>>>
>>>>
>> org.apache.asterix.test.aql.TestExecutor.executeUpdate(TestExecutor.java:331)
>>>> at
>>>>
>> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:496)
>>>> at
>>>>
>> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:762)
>>>> at
>>>>
>> org.apache.asterix.test.runtime.ExecutionTest.test(ExecutionTest.java:106)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>>
>>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> at
>>>>
>>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:497)
>>>> at
>>>>
>>>>
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>>>> at
>>>>
>>>>
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>>>> at
>>>>
>>>>
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>>>> at
>>>>
>>>>
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>>>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
>>>> at
>>>>
>>>>
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
>>>> at
>>>>
>>>>
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
>>>> at 
>>>> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
>>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
>>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
>>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
>>>> at org.junit.runners.Suite.runChild(Suite.java:127)
>>>> at org.junit.runners.Suite.runChild(Suite.java:26)
>>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
>>>> at 
>>>> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
>>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
>>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
>>>> at
>>>>
>>>>
>> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>>>> at
>>>>
>> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
>>>> at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
>>>> at
>>>>
>>>>
>> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119)
>>>> at
>>>>
>>>>
>> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
>>>> at
>>>>
>>>>
>> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
>>>> at
>>>> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>>
>>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> at
>>>>
>>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:497)
>>>> at
>>>> com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
>>>> ...Unexpected!
>>>>
>>>> On Sat, Jul 2, 2016 at 7:23 PM, Raman Grover
>>>> <ramangrover29@gmail.com>
>>>> wrote:
>>>>
>>>>> i am missing the attachment
>>>>> On Jul 2, 2016 6:39 PM, "Xikui Wang" <xikuiw@uci.edu> wrote:
>>>>>
>>>>>> Hi Raman,
>>>>>>
>>>>>> Thanks for your help. I tried this quick fix on my branch, but it
>>>>>> introduces some new exceptions. I think this causes Asterix fails
>>>>>> at
>>>>>> entering the external function. The error message is attached.
>>>>>>
>>>>>> Best,
>>>>>> Xikui
>>>>>>
>>>>>> On Fri, Jul 1, 2016 at 10:11 AM, Raman Grover
>>>>>> <ramangrover29@gmail.com
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Operations related to setting up an external library are 
>>>>>>> contained
>>>>>>> in
>>>>>>> ExternalLibraryUtil
>>>>>>> <
>>>>>>>
>>>>>
>>>>
>> https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
>>>>>>>>
>>>>>>>
>>>>>>> At line 382, we have
>>>>>>> // create and return the class loader
>>>>>>>
>>>>>>> ClassLoader classLoader = new URLClassLoader(urls,
>>>>>>> parentClassLoader);
>>>>>>> return classLoader;
>>>>>>>
>>>>>>> Above, we have the parentClassLoader set to the classloader for
>>>>>>> ExternalLibraryUtil which is the application class loader
>>>>>>> (AsterixDB's
>>>>>>> classloader that loads the dependencies from pom.xml). The
>>>>>>> proposed
>>>>>>> solution (a) in earlier thread - skipping application 
>>>>>>> classloader
>>>> would
>>>>>>> translate to replacing the above code with
>>>>>>>
>>>>>>> ClassLoader classLoader = new URLClassLoader(urls, null);
>>>>>>>
>>>>>>> Regards,
>>>>>>> Raman
>>>>>>>
>>>>>>> On Thu, Jun 30, 2016 at 4:41 PM, Xikui Wang <xikuiw@uci.edu>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi Abdullah,
>>>>>>>>
>>>>>>>> I reverted my code to reproduce the problem. Noticing this
>>>>>>>> external
>>>>>>>> function has a couple of other bugs but the dependency one is
>>>> blocking
>>>>>>>> others, so this should be enough to reproduce the problem.
>>>>>>>>
>>>>>>>>>
>>>>>>>> The external function package is geoTag.zip.
>>>>>>>>
>>>>>>>>> ​
>>>>>>>>>
>>>>>>>>> ​Test scripts are in tweetGeoTag.zip
>>>>>>>>
>>>>>>>> External function is loading data from data/, i.e.:
>>>>>>>> data/state.json
>>>> .
>>>>> So
>>>>>>>> all json files in data.zip need to be placed under
>>>>>>>> ../asterixdb/asterix-app/data/
>>>>>>>>
>>>>>>>> The real_tweets_adm.adm used in ddl is also attached.
>>>>>>>>
>>>>>>>> This setting will cause
>>>>>>>>
>>>>>>>> java.lang.NoSuchMethodError:
>>>>>>>>> com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
>>>>>>>>> at
>>>>>>>>>
>>>>>>>
>>>>>
>> com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:541)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>
>>>>>
>> com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:452)
>>>>>>>>> at
>>>>> org.wololo.geojson.GeoJSONFactory.<clinit>(GeoJSONFactory.java:17)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>> edu.uci.ics.cloudberry.gnosis.USGeoJSONIndex.loadShape(IGeoIndex.scala:29)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$.loadShape(USGeoGnosis.scala:101)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:20)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:18)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
>>>>>>>>> at scala.collection.immutable.List.foreach(List.scala:381)
>>>>>>>>> at
>>>>>>> scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
>>>>>>>>> at scala.collection.immutable.List.map(List.scala:285)
>>>>>>>>> at
>>>>>>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.load(USGeoGnosis.scala:18)
>>>>>>>>> at
>>>>>>>
>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.<init>(USGeoGnosis.scala:15)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFunction.initialize(GeoTagFunction.java:34)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>> org.apache.asterix.external.library.ExternalFunction.initialize(ExternalFunction.java:113)
>>>>>>>>>
>>>>>>>>
>>>>>>>> The attachments exceed the size limits. I moved them under this
>>>> google
>>>>>>>> drive directory:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>> https://drive.google.com/folderview?id=0B_6Dzy3OTjaNRUVJWTRvWEtKSU0&usp=sharing
>>>>>>>>
>>>>>>>> Let me know if you need anything else. Thanks.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Xikui
>>>>>>>>
>>>>>>>> On Thu, Jun 30, 2016 at 12:06 PM, abdullah alamoudi <
>>>>> bamousaa@gmail.com
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I will look into that.
>>>>>>>>>>
>>>>>>>>>> @Xikui,
>>>>>>>>>> Do you have a reproducable use case where such a behavior can
>>>>>>>>>> be
>>>>>>>> observed?
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Abdullah.
>>>>>>>>>>
>>>>>>>>>> On Thu, Jun 30, 2016 at 7:53 PM, Raman Grover <
>>>>>>> ramangrover29@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi
>>>>>>>>>>>
>>>>>>>>>>> Each external library has an separately associated custom
>>>>>>>>>>> class
>>>>>>> loader
>>>>>>>>>>> (URLClassLoader) that loads the contained functions and any
>>>>>>>> dependencies
>>>>>>>>>>> (jar files) packaged inside the library. The custom loader
>>>>>>>>>>> fits
>>>>>>> into
>>>>>>>> the
>>>>>>>>>>> natural hierarchy with system (classpath based) classloader 
>>>>>>>>>>> as
>>>>> the
>>>>>>>>>>> immediate parent and bootstrap classloader as the root. As 
>>>>>>>>>>> the
>>>>>>>>>> delegation
>>>>>>>>>>> model works, a class loader attempts to load a class only
>>>>>>>>>>> after
>>>>>>>>>> parent(s)
>>>>>>>>>>> in the lineage have failed.
>>>>>>>>>>>
>>>>>>>>>>> In your specific case, the system class loader is able to 
>>>>>>>>>>> find
>>>>> the
>>>>>>>>>> jackson
>>>>>>>>>>> library from the configured classpath and so the library
>>>>>>> classloader
>>>>>>>> is
>>>>>>>>>> not
>>>>>>>>>>> involved at all for this jar. The problem arising due to
>>>>> different
>>>>>>>>>> versions
>>>>>>>>>>> of a jar present across the hierarchy of classloaders is
>>>> similar
>>>>> to
>>>>>>>> one
>>>>>>>>>>> faced in web containers such as Tomcat or application 
>>>>>>>>>>> servers
>>>>> such
>>>>>>> as
>>>>>>>>>>> Websphere where system loaded classes can override any
>>>>> application
>>>>>>>>>> specific
>>>>>>>>>>> classes due to name conflict.
>>>>>>>>>>>
>>>>>>>>>>> The way you are defining a library is correct. The fix needs
>>>>>>>>>>> to
>>>>> be
>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>> way the classes are loaded by AsterixDB (here acting as an
>>>>>>> application
>>>>>>>>>>> server).
>>>>>>>>>>>
>>>>>>>>>>> I see two ways:-
>>>>>>>>>>>
>>>>>>>>>>> a) Skipping the system class loader: AsterixDB uses a
>>>>>>> URLClassloader
>>>>>>>> to
>>>>>>>>>>> load libraries. Setting the  parent of the this classloader 
>>>>>>>>>>> as
>>>>> null
>>>>>>>>>> implies
>>>>>>>>>>> the bootrstrap classloader (root) as the immediate parent. 
>>>>>>>>>>> The
>>>>>>>> bootstrap
>>>>>>>>>>> classloader loads rt.jar (java runtime classes) and i18n.jar
>>>> and
>>>>>>> these
>>>>>>>>>>> definitely do not have conflicts with your library
>>>>>>>>>>> dependencies
>>>>> (if
>>>>>>>> they
>>>>>>>>>>> do, not sure why one would override the classes in rt.jar 
>>>>>>>>>>> and
>>>> so
>>>>> we
>>>>>>>>>> need to
>>>>>>>>>>> fix the library). Note that the system classloader is out of
>>>> the
>>>>>>>>>> hierarchy
>>>>>>>>>>> here and so any other jars loaded by AsterixDB from its
>>>> classpath
>>>>>>>>>> (pom.xml)
>>>>>>>>>>> would not be visible.
>>>>>>>>>>>
>>>>>>>>>>> b) Defining a Custom Class loader: Inside AsterixDB, we
>>>> provide a
>>>>>>>> custom
>>>>>>>>>>> implementation of classloader that attempts to load a class
>>>> prior
>>>>>>> to
>>>>>>>>>>> delegating to the parent. This way, the library classes and
>>>>>>> packaged
>>>>>>>>>>> dependencies override any system level classes from the 
>>>>>>>>>>> class
>>>>> path
>>>>>>> and
>>>>>>>>>> even
>>>>>>>>>>> the classes contained in rt.jar and i18n.jar.
>>>>>>>>>>>
>>>>>>>>>>> I am opening the discussion here to suggest further
>>>> alternatives
>>>>> or
>>>>>>>>>> provide
>>>>>>>>>>> preferences.
>>>>>>>>>>>
>>>>>>>>>>> I have a preference for (a)  (skipping the system class
>>>>>>>>>>> loader)
>>>>> for
>>>>>>>> two
>>>>>>>>>>> reasons:
>>>>>>>>>>>
>>>>>>>>>>> a) it is simpler
>>>>>>>>>>>
>>>>>>>>>>> b) the other option allows a custom class loader to override
>>>>>>> classes
>>>>>>>> in
>>>>>>>>>>> rt.jar., which is ok but not how classloaders are supposed 
>>>>>>>>>>> to
>>>>> work
>>>>>>> in
>>>>>>>>>>> principle.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Raman
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Jun 29, 2016 11:07 PM, "Mike Carey" <dtabass@gmail.com>
>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Any classloader experts have suggestions...?
>>>>>>>>>>>> On Jun 29, 2016 10:26 PM, "Xikui Wang" <xikuiw@uci.edu>
>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Devs,
>>>>>>>>>>>>>
>>>>>>>>>>>>> We found a problem when trying to build external functions
>>>>> for
>>>>>>> the
>>>>>>>>>>>>> cloudberry demo.
>>>>>>>>>>>>>
>>>>>>>>>>>>> When the external function is depend on certain library,
>>>> the
>>>>>>>> library
>>>>>>>>>>> that
>>>>>>>>>>>>> comes with the external function will be blocked by same
>>>>>>> library
>>>>>>>> in
>>>>>>>>>>>>> AsterixDB. In our case, our external function 'geoTag' 
>>>>>>>>>>>>> uses
>>>>>>>> jackson
>>>>>>>>>>>> v2.7.1,
>>>>>>>>>>>>> and we packed all dependencies into one single jar. When
>>>>>>> running
>>>>>>>>>>> 'geoTag'
>>>>>>>>>>>>> on Asterix, it will call jackson v2.0.0 in AsterixDB which
>>>>>>> causes
>>>>>>>>>>>>> NullPointerException. We have to manually change pom.xml 
>>>>>>>>>>>>> in
>>>>>>>>>> AsterixDB
>>>>>>>>>>> to
>>>>>>>>>>>>> fix that.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We are wondering is that because we load the external
>>>>> function
>>>>>>> in
>>>>>>>> a
>>>>>>>>>>> wrong
>>>>>>>>>>>>> way, or this could be one possible interesting problem
>>>> which
>>>>> is
>>>>>>>>>> worth
>>>>>>>>>>>>> noticing. Thanks.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Xikui
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Raman
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>

Mime
View raw message