asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yingyi Bu <buyin...@gmail.com>
Subject Re: Nested type + open-enforced-index question.
Date Fri, 14 Jul 2017 23:09:43 GMT
>> When we are encounter a field (“nested”) for which the is no
compile-time information
>> we should assume that the type of this field is completely open, i.e.,
{}, and pass it down the chain.

Correct, since it's enforced.
The augmented enforced type maps should be recursively added into those
nested ARecordType.

Best,
Yingyi


On Fri, Jul 14, 2017 at 12:13 AM, Ildar Absalyamov <
ildar.absalyamov@gmail.com> wrote:

> However, there should be a way to deal with this issue when the top-level
> type is open.
>
> create type DBLPType as open {id: int32}
> create index title_index_DBLP on DBLP(nested.one.title: string?) enforced;
>
> When we are encounter a field (“nested”) for which the is no compile-time
> information we should assume that the type of this field is completely
> open, i.e., {}, and pass it down the chain.
>
> > On Jul 14, 2017, at 00:09, Ildar Absalyamov <ildar.absalyamov@gmail.com>
> wrote:
> >
> > Taewoo,
> >
> > You’ve correctly identified the issue here: to make use of an enforced
> index we must cast the record to a particular type, which is imposed by the
> index.
> >
> > So, using your example, if we have an index on path “nested.one.title”
> the indexed record must be castable to {…, “nested”: {…,”one”: {…,”title”:
> string, …}, ...},…}.
> > As you have observed a case when there is no “nested” field in the
> top-level type leads to exception, because it relies of a fact that there
> is a compile-time type information for a field “nested”. This type
> information is used to build a type for aforementioned cast operator.
> > Form the perspective of current implementation a runtime exception is a
> bug, instead it should have caught this issue during compile time.
> >
> >> On Jul 13, 2017, at 23:10, Taewoo Kim <wangsaeu@gmail.com> wrote:
> >>
> >> @Yingyi: thanks.
> >>
> >> @Mike: Yeah. My problem is how to associate the field type information.
> >> Ideally, the leaf level has the field to type hash map and the parent
> of it
> >> has that hashmap in its record type. And its parent needs to have the
> >> necessary information to reach to this record type. If we don't need any
> >> pre-defined type at each level to create a multi-level enforced index,
> then
> >> things will become more complex to me. :-) Anyway, we can discuss
> further
> >> to finalize the field type propagation implementation.
> >>
> >> Best,
> >> Taewoo
> >>
> >> On Thu, Jul 13, 2017 at 11:02 PM, Mike Carey <dtabass@gmail.com> wrote:
> >>
> >>> Taewoo,
> >>>
> >>> To clarify further what should work:
> >>> - We should support nested indexes that go down multiple levels.
> >>> - We should (ideally) support their use in index-NL joins.
> >>>
> >>> Reflecting on our earlier conversation(s), I think I can see why you're
> >>> asking this. :-) The augmented type information that'll be needed to do
> >>> this completely/properly will actually have to associate types with
> field
> >>> paths (not just with fields by name) - which is a slightly more
> complicated
> >>> association.
> >>>
> >>> Cheers,
> >>> Mike
> >>>
> >>>
> >>> On 7/13/17 10:54 PM, Yingyi Bu wrote:
> >>>
> >>>> Hi Taewoo,
> >>>>
> >>>> The first query shouldn't fail because indexnl is just a hint.
> >>>> The second query should succeed because it's a valid indexing
> statement.
> >>>> High nesting levels in open record like JSON is not uncommon.
> >>>>
> >>>> Best,
> >>>> Yingyi
> >>>>
> >>>>
> >>>> On Thu, Jul 13, 2017 at 10:51 PM, Taewoo Kim <wangsaeu@gmail.com>
> wrote:
> >>>>
> >>>> @Mike: In order to properly deal with the enforced index on a
> nested-type
> >>>>> field, I need to make sure that whether my understanding (each nested
> >>>>> type
> >>>>> (except the leaf level0 has a record type for the next level) is
> correct
> >>>>> or
> >>>>> not. Which one is a bug? The first one (without index) should fail?
> Or
> >>>>> the
> >>>>> second one (with an index) should succeed?
> >>>>>
> >>>>> Best,
> >>>>> Taewoo
> >>>>>
> >>>>> On Thu, Jul 13, 2017 at 9:58 PM, Yingyi Bu <buyingyi@gmail.com>
> wrote:
> >>>>>
> >>>>> Indeed, it's a bug!
> >>>>>>
> >>>>>> Best,
> >>>>>> Yingyi
> >>>>>>
> >>>>>> On Thu, Jul 13, 2017 at 9:52 PM, Mike Carey <dtabass@gmail.com>
> wrote:
> >>>>>>
> >>>>>> Sounds like a bug to me.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On 7/13/17 7:59 PM, Taewoo Kim wrote:
> >>>>>>>
> >>>>>>> Currently, I am working on a field type propagation without
using
> >>>>>>>> initializing the OptimizableSubTree in the current index
access
> >>>>>>>>
> >>>>>>> method.
> >>>>>
> >>>>>> I
> >>>>>>
> >>>>>>> am encountering an issue with an open-type enforced index.
So, I
> just
> >>>>>>>>
> >>>>>>> want
> >>>>>>
> >>>>>>> to make sure that my understanding is correct. It looks
like we
> can't
> >>>>>>>>
> >>>>>>> have
> >>>>>>
> >>>>>>> an enforced-index on a completely schemaless nested field.
For
> >>>>>>>>
> >>>>>>> example,
> >>>>>
> >>>>>> the
> >>>>>>>> following doesn't generate any issue.
> >>>>>>>>
> >>>>>>>> //
> >>>>>>>> create type DBLPType as open {id: int32}
> >>>>>>>> create type CSXType as closed {id: int32}
> >>>>>>>>
> >>>>>>>> create dataset DBLP(DBLPType) primary key id;
> >>>>>>>> create dataset CSX(CSXType) primary key id;
> >>>>>>>>
> >>>>>>>> for $a in dataset('DBLP')
> >>>>>>>> for $b in dataset('CSX')
> >>>>>>>> where $a.nested.one.title /*+ indexnl */ = $b.nested.one.title
> >>>>>>>> return {"arec": $a, "brec": $b}
> >>>>>>>> //
> >>>>>>>>
> >>>>>>>> However, the following generates an exception. So, can
we assume
> that
> >>>>>>>>
> >>>>>>> to
> >>>>>
> >>>>>> create an enforced-index, except the leaf level, there should
be a
> >>>>>>>>
> >>>>>>> defined
> >>>>>>
> >>>>>>> record type. For example, for this example, there should
be
> "nested"
> >>>>>>>>
> >>>>>>> type
> >>>>>>
> >>>>>>> and "one" type.
> >>>>>>>>
> >>>>>>>> //
> >>>>>>>> create type DBLPType as open {id: int32}
> >>>>>>>> create type CSXType as closed {id: int32}
> >>>>>>>>
> >>>>>>>> create dataset DBLP(DBLPType) primary key id;
> >>>>>>>> create dataset CSX(CSXType) primary key id;
> >>>>>>>>
> >>>>>>>> create index title_index_DBLP on DBLP(nested.one.title:
string?)
> >>>>>>>>
> >>>>>>> enforced;
> >>>>>>
> >>>>>>> create index title_index_CSX on CSX(nested.one.title: string?)
> >>>>>>>>
> >>>>>>> enforced;
> >>>>>
> >>>>>> for $a in dataset('DBLP')
> >>>>>>>> for $b in dataset('CSX')
> >>>>>>>> where $a.nested.one.title /*+ indexnl */ = $b.nested.one.title
> >>>>>>>> return {"arec": $a, "brec": $b}
> >>>>>>>> //
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Taewoo
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>
> >
> > Best regards,
> > Ildar
> >
>
> Best regards,
> Ildar
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message