asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Carey <>
Subject Re: Question about open indexes
Date Wed, 23 Sep 2015 01:33:11 GMT
Sounds kinda bad!  Also, I wonder what happens when the compiler 
encounters records in the dataset - whose type in the catalog doesn't 
claim to have a given (but now indexed) open field - e.g., during a data 
scan or an access via some other path?  Can Bad Things Happen due to the 
compiler not properly anticipating the casted form of the records?  
(Maybe I am misunderstanding something, but we should probably take a 
careful look at the test cases - and make sure we do things like add a 
bunch of records, then add such an index, then add some more records, 
then stress-test type-related things that come at the dataset (i) thru 
the index, (ii) thru a primary dataset scan, and (iii) thru some other 

On 9/22/15 4:06 PM, Taewoo Kim wrote:
> I think this issue: is
> related. Currently, index entries (SK, PK) are not deleted on an open-type
> secondary index during a deletion. This issue was not surfaced due to the
> fact that every search after a secondary index search had to go through the
> primary index lookup.
> Best,
> Taewoo
> On Tue, Sep 22, 2015 at 12:04 AM, Ildar Absalyamov <
>> wrote:
>> Abdullah,
>> If I remember correctly whenever a secondary open index is created all
>> existing records would be casted to a proper type to ensure that the index
>> creation is valid.
>> As for the overall correctness of casting operation, semantically creating
>> an open index is the same thing as altering the dataset type. The current
>> implementation allows only one open index of particular type created on a
>> single field. If we would have had “alter datatype” functionality the open
>> indexing would not be required at all.
>>> On Sep 21, 2015, at 23:25, abdullah alamoudi <> wrote:
>>> More thoughts:
>>> I assume the intention of the cast was just to make sure if the open
>> field
>>> exists, it is of the specified type. Moreover, the un-casted record
>> should
>>> be inserted into the index.
>>> If my assumptions are not correct, please, let me know ASAP.
>>> I have two thoughts on this:
>>> 1. Actually, insert plans show that the records being inserted into the
>>> primary index is actually the casted record creating the issue described
>>> above.
>>> 2. I don't believe this is the right way to ensure that the open field if
>>> exists is of the right type. why not extract the field using field access
>>> by name function and then verify the type using the field tag?
>>> On Tue, Sep 22, 2015 at 9:11 AM, abdullah alamoudi <>
>>> wrote:
>>>> Hi Dev, @Ildar,
>>>> In the insert pipeline for datasets with open indexes, we introduce a
>> cast
>>>> function before the insert and so one would expect the records to look
>> like
>>>> the casted record type which I assume has {{the closed fields + a
>> nullable
>>>> field}}.
>>>> The question is, what happens to the previously existing records?, since
>>>> now the index has both, records of the original type and records of the
>>>> casted type.
>>>> Thanks,
>>>> Abdullah.
>> Best regards,
>> Ildar

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message