asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Maxon <ima...@uci.edu>
Subject Re: Severe issue relating to indexing on nullable fields within a closed type?
Date Tue, 09 Jun 2015 17:12:14 GMT
My quack theory is that it is due to the RMI somehow, or that the RMI
reveals it. It isn't specific to managix. It happens as long as the CC and
NC reside in different JVMs.

- Ian

On Tue, Jun 9, 2015 at 10:03 AM, Ildar Absalyamov <
ildar.absalyamov@gmail.com> wrote:

> As Ian mentioned test case already exists (which I thought was good enough
> to prevent regressions like this one), but for some reason it’s not
> triggering the error, whereas if the same ddl\query is executed on a local
> cluster, which was prepared by managix, the error will trigger.
>
> > On Jun 9, 2015, at 08:07, Mike Carey <dtabass@gmail.com> wrote:
> >
> > Ildar, can you try to nail this one down and also add a test case if you
> > can create one once this is making more sense...?
> > On Jun 8, 2015 11:58 PM, "Ildar Absalyamov" <ildar.absalyamov@gmail.com>
> > wrote:
> >
> >> Interesting, I guess step 1 here would be to investigate what’s
> different
> >> in test framework\AsterixHyracksIntegrationUntil setup and regular
> cluster
> >> configuration, created by managix.
> >>
> >>> On Jun 8, 2015, at 23:37, Ian Maxon <imaxon@uci.edu> wrote:
> >>>
> >>> I was playing around with the latest snapshot version, trying to
> create a
> >>> fresher version of my demo docker container, and so I ran through the
> >>> ADM/AQL 101. However I couldn't get it to work, which was really
> >> surprising
> >>> (and distressing). The error appears after trying to create an RTree
> >> index
> >>> on
> >>> sender-location:
> >>>
> >>>
> >>> SEVERE: java.lang.NullPointerException
> >>>> edu.uci.ics.asterix.metadata.MetadataException:
> >>>> java.lang.NullPointerException
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.metadata.MetadataNode.addIndex(MetadataNode.java:229)
> >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>>> at
> >>>>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >>>> at
> >>>>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>>> at java.lang.reflect.Method.invoke(Method.java:497)
> >>>> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
> >>>> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> >>>> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> >>>> at java.security.AccessController.doPrivileged(Native Method)
> >>>> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> >>>> at
> >> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
> >>>> at
> >>>>
> >>
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
> >>>> at
> >>>>
> >>
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$97(TCPTransport.java:683)
> >>>> at
> >>>>
> >>
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/1502895914.run(Unknown
> >>>> Source)
> >>>> at java.security.AccessController.doPrivileged(Native Method)
> >>>> at
> >>>>
> >>
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
> >>>> at
> >>>>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>>> at
> >>>>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>>> at java.lang.Thread.run(Thread.java:745)
> >>>> at
> >>>>
> >>
> sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)
> >>>> at
> >>>>
> >>
> sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)
> >>>> at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
> >>>> at
> >>>>
> >>
> java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:194)
> >>>> at
> >>>>
> >>
> java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:148)
> >>>> at com.sun.proxy.$Proxy11.addIndex(Unknown Source)
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.metadata.MetadataManager.addIndex(MetadataManager.java:417)
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.aql.translator.AqlTranslator.handleCreateIndexStatement(AqlTranslator.java:920)
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.aql.translator.AqlTranslator.compileAndExecute(AqlTranslator.java:261)
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:97)
> >>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
> >>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
> >>>> at
> >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:970)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:904)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
> >>>> at org.eclipse.jetty.server.Server.handle(Server.java:347)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:439)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:924)
> >>>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:781)
> >>>> at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:43)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:545)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:43)
> >>>> at
> >>>>
> >>
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
> >>>> at java.lang.Thread.run(Thread.java:745)
> >>>> Caused by: java.lang.NullPointerException
> >>>> at java.io.DataOutputStream.writeUTF(DataOutputStream.java:347)
> >>>> at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323)
> >>>> at
> >>>>
> >>
> edu.uci.ics.hyracks.dataflow.common.data.marshalling.UTF8StringSerializerDeserializer.serialize(UTF8StringSerializerDeserializer.java:44)
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.dataflow.data.nontagged.serde.AStringSerializerDeserializer.serialize(AStringSerializerDeserializer.java:47)
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.dataflow.data.nontagged.serde.AStringSerializerDeserializer.serialize(AStringSerializerDeserializer.java:26)
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.formats.nontagged.AqlSerializerDeserializerProvider$1.serialize(AqlSerializerDeserializerProvider.java:208)
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.formats.nontagged.AqlSerializerDeserializerProvider$1.serialize(AqlSerializerDeserializerProvider.java:189)
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.metadata.entitytupletranslators.IndexTupleTranslator.getTupleFromMetadataEntity(IndexTupleTranslator.java:260)
> >>>> at
> >>>>
> >>
> edu.uci.ics.asterix.metadata.MetadataNode.addIndex(MetadataNode.java:224)
> >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>>> at
> >>>>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >>>> at
> >>>>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>>> at java.lang.reflect.Method.invoke(Method.java:497)
> >>>> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
> >>>> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> >>>> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> >>>> at java.security.AccessController.doPrivileged(Native Method)
> >>>> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> >>>> at
> >> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
> >>>> at
> >>>>
> >>
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
> >>>> at
> >>>>
> >>
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$97(TCPTransport.java:683)
> >>>> at
> >>>>
> >>
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/1502895914.run(Unknown
> >>>> Source)
> >>>> at java.security.AccessController.doPrivileged(Native Method)
> >>>> at
> >>>>
> >>
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
> >>>> at
> >>>>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>>> at
> >>>>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>>> ... 1 more
> >>>
> >>>
> >>> I started going back, trying this on older and older revisions until it
> >>> stopped happening. It looks like this somehow has been lurking in the
> >>> Open/nested indexing code for a long time. The error does not reproduce
> >> in
> >>> the testing framework, for some reason I don't understand.
> >>>
> >>> The source of the bug seems to be that the RTree index is on a type of
> >>> UNION(sender-location, null). The name of the type is not set like it
> >>> should be, it is null. Then in IndexTupleTranslator, the 9th field of
> the
> >>> new metadata record is written with the name of the type- which is
> null,
> >> so
> >>> everything explodes.
> >>>
> >>> What is especially puzzling- is that this doesn't reproduce in
> >>> AsterixHyracksIntegrationUtil. There, the name of the type is
> >>> "Field_sender-location_in_FacebookMessageType". This type is created
> too
> >> in
> >>> the deployed version, however it doesn't seem to be picked up.
> >>>
> >>> I will keep digging on this, but I imagine someone more familiar with
> the
> >>> Open/Nested indexing patch might have an idea about why this is
> >> happening.
> >>> To reproduce just try deploying the snapshot version from the website
> >>> normally on your machine, and run through ADM/AQL 101.
> >>>
> >>> - Ian
> >>
> >> Best regards,
> >> Ildar
> >>
> >>
>
> Best regards,
> Ildar
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message