asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Carey <dtab...@gmail.com>
Subject Re: Status of the Metadata fixes (https://asterix-gerrit.ics.uci.edu/#/c/323/)
Date Wed, 05 Aug 2015 06:46:42 GMT
Hmmm - I think I understand...  I think we need to remove the ? from the 
create index syntax (it isn't something it makes any sense to allow 
users to say) and then we should make sure at create time that the 
indexed field or fields are NOT a part of the known schema of the target 
dataset.  Right?

On 8/4/15 11:25 PM, Ildar Absalyamov wrote:
> OK, so I will include an index-creation time check to avoid the nullable types in this
case then.
>
>> On Aug 4, 2015, at 22:35, Mike Carey <dtabass@gmail.com> wrote:
>>
>> Oh - I missed the point on this - WIERD!  This seems like a "bug" - i.e., I don't
think ? should be part of the syntax either, i.e., I agree with Ildar that this makes no sense
- I'm not sure why it's there.  Let's get rid of that and then we won't need this bit of metadata
either.  (Though we'll need the code changes to fix this.)
>>
>> On 8/4/15 2:34 PM, Ildar Absalyamov wrote:
>>> As Till mentioned in the comment the problem is that we might need
>>> nullability information in two cases:
>>> 1) When a field is declared nullable in the schema. In this case the
>>> information is persisted into the "IsNullable" metadata field, introduced
>>> in the patch
>>> 2) When we are declaring an open index of a nullable type (which is a
>>> useless thing to do in my opinion). In this case right now we persist only
>>> the type name, thus a "?" marker is needed to deserialize the proper type
>>> back.
>>> The conclusion was to store nullability information in "IsNullable" field
>>> in the second case as well, which as I hoped will allow to reuse some code
>>> from schema serialization. However the format of the metadata record is
>>> slightly different in the case of an index. I did not invest that much time
>>> into the issue since the last week, was hoping to finish soon.
>>>
>>> My main concern is whether the second case is valid at all. When an open
>>> index is declared on the field it does not matter if the type of the index
>>> is nullable or not, since the field value could potentially be null by
>>> definition.
>>> However, as Till mentioned in our discussion, it might make a difference if
>>> we distinguish between the case when field "foo" has a value "null" and the
>>> case when "foo" is completely missing from the record. Thoughts?
>>>
>>> 2015-08-04 14:07 GMT-07:00 Till Westmann <tillw@apache.org>:
>>>
>>>> I’ve added a comment to the review about what I think is an open issue.
>>>> It would be nice to get more eyes/opinions on this to see if this is an
>>>> issue and should be addressed.
>>>>
>>>> Thanks,
>>>> Till
>>>>
>>>>> On Aug 4, 2015, at 1:53 PM, Steven Jacobs <sjaco002@ucr.edu> wrote:
>>>>>
>>>>> It still has my +1 (I reviewed the changes since patchset 3), but it's
>>>>> waiting for a +2 from Till.
>>>>> Steven
>>>>>
>>>>> On Tue, Aug 4, 2015 at 1:36 PM, Ian Maxon <imaxon@uci.edu> wrote:
>>>>>
>>>>>> Hi,
>>>>>> This change is the last feature on the checklist before release,
>>>>>> AFAIK. Just wanted to start a thread so there's visibility into the
>>>>>> status of it, as I think there's been things going on behind the
>>>>>> scenes. I believe right now it is under review, and that Till has
>>>>>> provided comments to Ildar. However I'm not sure what has been going
>>>>>> on after that. Will this patch need another round of fixes and review,
>>>>>> or are the comments  something that is addressable post-release
>>>>>> without a breaking metadata change? If it does need more work, what
is
>>>>>> the time frame for that?
>>>>>>
>>>>>> -Ian
> Best regards,
> Ildar
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message