asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ildar Absalyamov <ildar.absalya...@gmail.com>
Subject Re: First Apache incubating release - blockers?
Date Fri, 14 Aug 2015 06:24:51 GMT

> On Aug 13, 2015, at 23:07, abdullah alamoudi <bamousaa@gmail.com> wrote:
> 
> I assume you meant  “Dataset" record type definition and not "Datatype" and
> you're right. they are the same and can be elevated one level higher :-).

OK, I will move it as a part of this change in this case.

> 
> For everyone's information, here is a description of the problem:
> In our regular secondary indexes, the structure of the tuple consists of
> <SecondaryKeys,PrimaryKeys> and the Hyracks jobs get constructed based on
> this assumption. However, our Metadata secondary indexes don't follow this
> convention and so we end up with runtime errors whenever they are picked
> for access.

Correct, instead these indexes construct tuple in a form <SecondaryKey, StuffThatReallyNeeded>,
which allows them to avoid probing the primary for the needed information, returning via index-only
lookup.

> 
> Cheers,
> Abdullah.
> 
> On Fri, Aug 14, 2015 at 8:59 AM, Ildar Absalyamov <
> ildar.absalyamov@gmail.com> wrote:
> 
>> @Abdullah,
>> OK, now I see that change. IMHO this is a hack, but I will adopt that
>> approach that as well to submit the code review. However this issue really
>> needs to be addressed.
>> 
>> Side question, when you introduced external indexes you have added a field
>> “ExternalDetails” (which in turn is a nested record) into “Datatype" record
>> type definition in addition to existing “InternalDetails” field (BTW I
>> believe either of the fields exists at a particular point in time, maybe be
>> can use UNION for them?). However both of them have nested field
>> “GroupName”, which is used by one of the secondary metadata indexes. Is
>> that the same GroupName in both cases? If yes, can we move this field 1
>> level up from {External\Internal}Details to avoid duplication?
>> 
>>> On Aug 13, 2015, at 20:48, abdullah alamoudi <bamousaa@gmail.com> wrote:
>>> 
>>> Ildar,
>>> I remember that I had an issue with the secondary indexes of Metadata
>> when
>>> I enabled the prefix search for our indexes. I believe with the current
>>> structure of the secondary indexes, they are unusable in a typical query
>>> execution job and so I disabled secondary index access for Metadata
>> indexes.
>>> 
>>> I think we should stick to that at the moment at least. I also believe
>> that
>>> the point of the secondary indexes for Metadata is to validate ddl
>>> operations and not to be used for dataset access within Hyracks.
>>> 
>>> Cheers,
>>> Abdullah.
>>> 
>>> On Fri, Aug 14, 2015 at 12:10 AM, Ildar Absalyamov <
>>> ildar.absalyamov@gmail.com> wrote:
>>> 
>>>> End of this\early next week would be the estimate.
>>>> 
>>>> 2015-08-13 13:53 GMT-07:00 Mike Carey <dtabass@gmail.com>:
>>>> 
>>>>> Okay - cool - future improvements needed, clearly.
>>>>> Any ETA for the interim solution?
>>>>> Thx!
>>>>> Mike
>>>>> 
>>>>> 
>>>>> On 8/13/15 1:42 PM, Ildar Absalyamov wrote:
>>>>> 
>>>>>> With respect to metadata change I am fixing the issue with
>> bootstrapped
>>>>>> secondary metadata indexes.
>>>>>> Strictly speaking they are not indexes per se, more similar to
>>>>>> materialized
>>>>>> views.
>>>>>> Writing\reading to\from those indexes is done manually bypassing
the
>>>>>> regular index insert\delete codepath. This is really bad from
>> dogfooding
>>>>>> perspective, because it breaks the type validation logic when
>> Select\INL
>>>>>> Join rewrite rules are trying to fire.
>>>>>> However I have an idea how to use them without breaking the
>> validation
>>>>>> logic.
>>>>>> 
>>>>>> 2015-08-13 13:11 GMT-07:00 Jochen Wiedmann <jochen.wiedmann@gmail.com
>>> :
>>>>>> 
>>>>>> On Thu, Aug 13, 2015 at 10:08 PM, Mike Carey <dtabass@gmail.com>
>> wrote:
>>>>>>> 
>>>>>>>> So where are we on feeds and metadata - the release blockers?
>>>>>>>> (@Raman?  Feed tutorial and API key?  @Ildar?  Your changes?)
>>>>>>>> 
>>>>>>> I'd suggest to complete the "Verify distribution rights" section
on
>>>>>>> http://incubator.apache.org/projects/asterixdb.html, before cutting
>> a
>>>>>>> release.
>>>>>>> 
>>>>>>> Jochen
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Any world that can produce the Taj Mahal, William Shakespeare,
>>>>>>> and Stripe toothpaste can't be all bad. (C.R. MacNamara, One
Two
>> Three)
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> Best regards,
>>>> Ildar
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Amoudi, Abdullah.
>> 
>> Best regards,
>> Ildar
>> 
>> 
> 
> 
> -- 
> Amoudi, Abdullah.

Best regards,
Ildar


Mime
View raw message