asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Carey <dtab...@gmail.com>
Subject Re: Severe issue relating to indexing on nullable fields within a closed type?
Date Sat, 13 Jun 2015 08:02:59 GMT
Agreed.  The upcoming release will be major, so it is kind of our
one chance to mess with the metadata in such a major way....
@ildar, I think you might be the most nimble at this particular
set of issues, given your experience in this part of the code...?
(This is probably a release blocker for all the feed, external data,
nested indexing, and open indexing goodies....)

On 6/12/15 6:43 PM, Ian Maxon wrote:
> More discussion was had today about this, to summarize I think these
> were the actions decided on in order of greatest to least priority:
>
> - Hotfix the current manifestation of the issue (done and merged)
> - Change the metadata to not give optional fields in closed types an
> anonymous type name (this will break current datasets). Would be
> highly desirable to do this before Apache release in the next week or
> so.
> - Be sure that no modification occurs on MD insert, i.e. treat the
> inserted object as final.
>
>
> Thoughts?
>
> -Ian
>
> On Thu, Jun 11, 2015 at 5:28 PM, Ian Maxon <imaxon@uci.edu> wrote:
>>>> The locally produced Datatype object is actually never totally valid though
>>>> from the way I am seeing things. So there is no reason to cache it in the
>>>> first place.
>>>>
>>> Indeed. But bypassing a cache is valid in this particular case only.
>>>
>> I think the crux of the problem though is that the MD node should
>> really be treating any object it receives to insert as final. Having
>> the client generate something to be inserted, then having the MD node
>> mutate that upon asking it to store the information, seems like it is
>> asking for trouble. It's certainly possible to do a delicate dance of
>> cache invalidation, but unless there's a good reason to do that- why
>> shouldn't we avoid it?
>>
>> - Ian
>>
>>
>> On Thu, Jun 11, 2015 at 5:00 PM, Ildar Absalyamov
>> <ildar.absalyamov@gmail.com> wrote:
>>> Replies inline:
>>>
>>>> On Jun 11, 2015, at 16:28, Ian Maxon <imaxon@uci.edu> wrote:
>>>>
>>>> The locally produced Datatype object is actually never totally valid though
>>>> from the way I am seeing things. So there is no reason to cache it in the
>>>> first place.
>>>>
>>> Indeed. But bypassing a cache is valid in this particular case only.
>>>
>>>> I talked with Mike about this today. I think the main points were as
>>>> follows (correct me if I am wrong of course!):
>>>>
>>>> - Is it possible to move the name generation in DatatypeTupleTranslator
>>>> into TypeTranslator.computeTypes? This would let the client have the fully
>>>> updated object and avoid updating anything at the MD node side.
>>> I was thinking about this as well. Don’t quite see why that would not be possible.
>>>
>>>> - The entire type name generation seems weird. Why is it that we need those
>>>> names in the first place? For example take this type:
>>>>
>>>>
>>>> create type FacebookMessageType as closed {
>>>>         message-id: int32,
>>>>         author-id: int32,
>>>>         in-response-to: int32?,
>>>>         sender-location: point?,
>>>>         message: string
>>>>     }
>>>>
>>>> and the Metadata entry for that type:
>>>>
>>>> { "DataverseName": "TinySocial",
>>>>    "DatatypeName": "FacebookMessageType",
>>>>    "Derived": { "Tag": "RECORD",
>>>>                       "IsAnonymous": false,
>>>>                       "EnumValues": null,
>>>>                       "Record": { "IsOpen": false,
>>>>                                         "Fields": [ { "FieldName":
>>>> "message-id", "FieldType": "int32" },
>>>>                                                         { "FieldName":
>>>> "author-id", "FieldType": "int32" },
>>>>                                                         { "FieldName":
>>>> "in-response-to",
>>>>                                                           "FieldType":
>>>> "Field_in-response-to_in_FacebookMessageType"
>>>>                                                          },
>>>>                                                          { "FieldName":
>>>> "sender-location",
>>>>                                                         "FieldType":
>>>> "Field_sender-location_in_FacebookMessageType"
>>>>                                                          },
>>>>                                                          { "FieldName":
>>>> "message", "FieldType": "string" }
>>>>                                                        ]
>>>>                                        },
>>>>                       "Union": null,
>>>>                       "UnorderedList": null,
>>>>                       "OrderedList": null
>>>>     },
>>>>    "Timestamp": "Thu Jun 11 11:35:33 PDT 2015"
>>>> }
>>>>
>>>> Wouldn't it be better if we could make the Field entries for the nullable
>>>> fields not dependent on this weird anonymous UNION(something,null) type?
>>>> Something more like
>>>>
>>>> { "FieldName": "in-response-to", "FieldType":"point", "Optional":true}
>>> Yeah, this whole naming thing might look weird, especially when it is applied
to nullable fields, which are in essence a primitive type.
>>> But for whatever reason we have chosen implementing a nullable fields as a a
complex data type: union with null field. At the same time generating names for complex types
is legitimate, and even required in case of anonymous nested types, e.g:
>>>
>>> create type FacebookMessageType as closed {
>>>         message-id: int32,
>>>         author-id: int32,
>>>         in-response-to: int32?,
>>>         sender-location: {
>>>                  coordinate:point,
>>>                  POI: string
>>>          }?,
>>>         message: string
>>>     }
>>>
>>>> We should definitely discuss this more on Friday
>>>>
>>>> - Ian
>>>>
>>>> On Wed, Jun 10, 2015 at 5:39 PM, Ildar Absalyamov <
>>>> ildar.absalyamov@gmail.com> wrote:
>>>>
>>>>> Just a bit of background, the bug is not specific to RTree, but will
be
>>>>> triggered on any index, created on a nullable field.
>>>>> In order to persist datatype with a nullable field a special internal
type
>>>>> with name ’Type_xxx_UnionType_yyy’ must be created on MD node side.
>>>>> However the stale version of datatype (where the name of the nullable
>>>>> field is not yet computed) is returned on the client, where it got cached.
>>>>>
>>>>> Clearly, relying on the fact that the argument supposedly passed by
>>>>> reference will get updated is wrong, when things are sent via RMI (and
>>>>> Ian’s patch resolves that).
>>>>> However I think a better way would be to invalidate locally cached stale
>>>>> copy, so when the actual index is created it will query the MD and get
the
>>>>> right Datatype.
>>>>> That is in line with the question Niki (our visiting student from Germany)
>>>>> asked me today, for which I did not had a clear answer: how do we manage
>>>>> the local MD cache and invalidate it?
>>>>>
>>>>>> On Jun 10, 2015, at 02:43, Ian Maxon <imaxon@uci.edu> wrote:
>>>>>>
>>>>>> Indeed, it is weird practice at best. The real index is an RTree
index on
>>>>>> FacebookMessages.sender-location, which is nullable, and the code
path to
>>>>>> create that index just happens to trigger this.
>>>>>>
>>>>>> On Tue, Jun 9, 2015 at 11:35 PM, Mike Carey <dtabass@gmail.com>
wrote:
>>>>>>
>>>>>>> This sounds like a bad practice (doing any mods on the MD node
- we
>>>>> should
>>>>>>> view the call as passing in immutable data).
>>>>>>> But I'm not sure which index is being referred to here - what
index on
>>>>>>> what dataset is the "actual index" below?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 6/9/15 5:11 PM, Ian Maxon wrote:
>>>>>>>
>>>>>>>> So I think I might have a fix for this:
>>>>>>>> https://asterix-gerrit.ics.uci.edu/#/c/283/1
>>>>>>>>
>>>>>>>> This bug is really insidious. What I think is happening is
that in
>>>>>>>> AqlTranslator, the types are computed and given to the MetadataManager
>>>>> as
>>>>>>>> a
>>>>>>>> new Datatype, to insert into the metadata node via RMI on
MetadataNode.
>>>>>>>> However MetadataNode modifies the object it is given before
it inserts
>>>>> it
>>>>>>>> into the actual index. That is where the names of the nullable
types
>>>>> get
>>>>>>>> generated, and so the MetadataManager doesn't get to see
those changes
>>>>>>>> unless MetadataManager and MetadataNode are actually sharing
the same
>>>>>>>> object. So if it is stale, that object gets inserted into
the cache,
>>>>> and
>>>>>>>> when the index is created, it sees the stale type information
rather
>>>>> than
>>>>>>>> the true value that's stored in the index.
>>>>>>>>
>>>>>>>> My fix is just to read it again before inserting into the
cache. I'm
>>>>> not
>>>>>>>> really sure thats the right thing to do though. And if there
are any
>>>>> other
>>>>>>>> instances of this type (caller expecting changes on remote
side to
>>>>>>>> reflect), the same sort of behavior can happen.
>>>>>>>>
>>>>>>>> - Ian
>>>>>>>>
>>>>>>>> On Tue, Jun 9, 2015 at 10:12 AM, Ian Maxon <imaxon@uci.edu>
wrote:
>>>>>>>>
>>>>>>>> My quack theory is that it is due to the RMI somehow, or
that the RMI
>>>>>>>>> reveals it. It isn't specific to managix. It happens
as long as the CC
>>>>>>>>> and
>>>>>>>>> NC reside in different JVMs.
>>>>>>>>>
>>>>>>>>> - Ian
>>>>>>>>>
>>>>>>>>> On Tue, Jun 9, 2015 at 10:03 AM, Ildar Absalyamov <
>>>>>>>>> ildar.absalyamov@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> As Ian mentioned test case already exists (which I thought
was good
>>>>>>>>>> enough to prevent regressions like this one), but
for some reason
>>>>> it’s
>>>>>>>>>> not
>>>>>>>>>> triggering the error, whereas if the same ddl\query
is executed on a
>>>>>>>>>> local
>>>>>>>>>> cluster, which was prepared by managix, the error
will trigger.
>>>>>>>>>>
>>>>>>>>>> On Jun 9, 2015, at 08:07, Mike Carey <dtabass@gmail.com>
wrote:
>>>>>>>>>>> Ildar, can you try to nail this one down and
also add a test case if
>>>>>>>>>>> you
>>>>>>>>>>> can create one once this is making more sense...?
>>>>>>>>>>> On Jun 8, 2015 11:58 PM, "Ildar Absalyamov" <
>>>>>>>>>>> ildar.absalyamov@gmail.com
>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Interesting, I guess step 1 here would be to
investigate what’s
>>>>>>>>>>> different
>>>>>>>>>>> in test framework\AsterixHyracksIntegrationUntil
setup and regular
>>>>>>>>>>> cluster
>>>>>>>>>>> configuration, created by managix.
>>>>>>>>>>>> On Jun 8, 2015, at 23:37, Ian Maxon <imaxon@uci.edu>
wrote:
>>>>>>>>>>>>> I was playing around with the latest
snapshot version, trying to
>>>>>>>>>>>>>
>>>>>>>>>>>> create a
>>>>>>>>>>> fresher version of my demo docker container,
and so I ran through
>>>>> the
>>>>>>>>>>>>> ADM/AQL 101. However I couldn't get it
to work, which was really
>>>>>>>>>>>>>
>>>>>>>>>>>> surprising
>>>>>>>>>>>>
>>>>>>>>>>>>> (and distressing). The error appears
after trying to create an
>>>>> RTree
>>>>>>>>>>>> index
>>>>>>>>>>>>
>>>>>>>>>>>>> on
>>>>>>>>>>>>> sender-location:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> SEVERE: java.lang.NullPointerException
>>>>>>>>>>>>>
>>>>>>>>>>>>>> edu.uci.ics.asterix.metadata.MetadataException:
>>>>>>>>>>>>>> java.lang.NullPointerException
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.metadata.MetadataNode.addIndex(MetadataNode.java:229)
>>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:497)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
>>>>>>>>>>> at sun.rmi.transport.Transport$1.run(Transport.java:200)
>>>>>>>>>>>>>> at sun.rmi.transport.Transport$1.run(Transport.java:197)
>>>>>>>>>>>>>> at java.security.AccessController.doPrivileged(Native
Method)
>>>>>>>>>>>>>> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$97(TCPTransport.java:683)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/1502895914.run(Unknown
>>>>>>>>>>> Source)
>>>>>>>>>>>>>> at java.security.AccessController.doPrivileged(Native
Method)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)
>>>>>>>>>>> at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:194)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:148)
>>>>>>>>>>> at com.sun.proxy.$Proxy11.addIndex(Unknown Source)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.metadata.MetadataManager.addIndex(MetadataManager.java:417)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.aql.translator.AqlTranslator.handleCreateIndexStatement(AqlTranslator.java:920)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.aql.translator.AqlTranslator.compileAndExecute(AqlTranslator.java:261)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:97)
>>>>>>>>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
>>>>>>>>>>>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:970)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:904)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
>>>>>>>>>>> at org.eclipse.jetty.server.Server.handle(Server.java:347)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:439)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:924)
>>>>>>>>>>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:781)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:43)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:545)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:43)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
>>>>>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>>>>>>>>> Caused by: java.lang.NullPointerException
>>>>>>>>>>>>>> at java.io.DataOutputStream.writeUTF(DataOutputStream.java:347)
>>>>>>>>>>>>>> at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.hyracks.dataflow.common.data.marshalling.UTF8StringSerializerDeserializer.serialize(UTF8StringSerializerDeserializer.java:44)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.dataflow.data.nontagged.serde.AStringSerializerDeserializer.serialize(AStringSerializerDeserializer.java:47)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.dataflow.data.nontagged.serde.AStringSerializerDeserializer.serialize(AStringSerializerDeserializer.java:26)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.formats.nontagged.AqlSerializerDeserializerProvider$1.serialize(AqlSerializerDeserializerProvider.java:208)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.formats.nontagged.AqlSerializerDeserializerProvider$1.serialize(AqlSerializerDeserializerProvider.java:189)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.metadata.entitytupletranslators.IndexTupleTranslator.getTupleFromMetadataEntity(IndexTupleTranslator.java:260)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> edu.uci.ics.asterix.metadata.MetadataNode.addIndex(MetadataNode.java:224)
>>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:497)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
>>>>>>>>>>> at sun.rmi.transport.Transport$1.run(Transport.java:200)
>>>>>>>>>>>>>> at sun.rmi.transport.Transport$1.run(Transport.java:197)
>>>>>>>>>>>>>> at java.security.AccessController.doPrivileged(Native
Method)
>>>>>>>>>>>>>> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$97(TCPTransport.java:683)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/1502895914.run(Unknown
>>>>>>>>>>> Source)
>>>>>>>>>>>>>> at java.security.AccessController.doPrivileged(Native
Method)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>>>>>>>> ... 1 more
>>>>>>>>>>>>> I started going back, trying this on
older and older revisions
>>>>> until
>>>>>>>>>>>> it
>>>>>>>>>>> stopped happening. It looks like this somehow
has been lurking in
>>>>> the
>>>>>>>>>>>>> Open/nested indexing code for a long
time. The error does not
>>>>>>>>>>>>>
>>>>>>>>>>>> reproduce
>>>>>>>>>>> in
>>>>>>>>>>>>> the testing framework, for some reason
I don't understand.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The source of the bug seems to be that
the RTree index is on a
>>>>> type
>>>>>>>>>>>>> of
>>>>>>>>>>>>> UNION(sender-location, null). The name
of the type is not set
>>>>> like it
>>>>>>>>>>>>> should be, it is null. Then in IndexTupleTranslator,
the 9th
>>>>> field of
>>>>>>>>>>>> the
>>>>>>>>>>> new metadata record is written with the name
of the type- which is
>>>>>>>>>>>> null,
>>>>>>>>>>> so
>>>>>>>>>>>>> everything explodes.
>>>>>>>>>>>>>
>>>>>>>>>>>>> What is especially puzzling- is that
this doesn't reproduce in
>>>>>>>>>>>>> AsterixHyracksIntegrationUtil. There,
the name of the type is
>>>>>>>>>>>>> "Field_sender-location_in_FacebookMessageType".
This type is
>>>>> created
>>>>>>>>>>>> too
>>>>>>>>>>> in
>>>>>>>>>>>>> the deployed version, however it doesn't
seem to be picked up.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I will keep digging on this, but I imagine
someone more familiar
>>>>> with
>>>>>>>>>>>> the
>>>>>>>>>>> Open/Nested indexing patch might have an idea
about why this is
>>>>>>>>>>>> happening.
>>>>>>>>>>>>
>>>>>>>>>>>>> To reproduce just try deploying the snapshot
version from the
>>>>> website
>>>>>>>>>>>>> normally on your machine, and run through
ADM/AQL 101.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Ian
>>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Ildar
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>> Ildar
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> Best regards,
>>>>> Ildar
>>>>>
>>>>>
>>> Best regards,
>>> Ildar
>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message