asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Taewoo Kim <wangs...@gmail.com>
Subject Re: Nested type + open-enforced-index question.
Date Fri, 14 Jul 2017 07:13:32 GMT
@Ildar: Yes. The current implementation requires that. So, I asked whether
which one makes sense.

Best,
Taewoo

On Fri, Jul 14, 2017 at 12:09 AM, Ildar Absalyamov <
ildar.absalyamov@gmail.com> wrote:

> Taewoo,
>
> You’ve correctly identified the issue here: to make use of an enforced
> index we must cast the record to a particular type, which is imposed by the
> index.
>
> So, using your example, if we have an index on path “nested.one.title” the
> indexed record must be castable to {…, “nested”: {…,”one”: {…,”title”:
> string, …}, ...},…}.
> As you have observed a case when there is no “nested” field in the
> top-level type leads to exception, because it relies of a fact that there
> is a compile-time type information for a field “nested”. This type
> information is used to build a type for aforementioned cast operator.
> Form the perspective of current implementation a runtime exception is a
> bug, instead it should have caught this issue during compile time.
>
> > On Jul 13, 2017, at 23:10, Taewoo Kim <wangsaeu@gmail.com> wrote:
> >
> > @Yingyi: thanks.
> >
> > @Mike: Yeah. My problem is how to associate the field type information.
> > Ideally, the leaf level has the field to type hash map and the parent of
> it
> > has that hashmap in its record type. And its parent needs to have the
> > necessary information to reach to this record type. If we don't need any
> > pre-defined type at each level to create a multi-level enforced index,
> then
> > things will become more complex to me. :-) Anyway, we can discuss further
> > to finalize the field type propagation implementation.
> >
> > Best,
> > Taewoo
> >
> > On Thu, Jul 13, 2017 at 11:02 PM, Mike Carey <dtabass@gmail.com> wrote:
> >
> >> Taewoo,
> >>
> >> To clarify further what should work:
> >> - We should support nested indexes that go down multiple levels.
> >> - We should (ideally) support their use in index-NL joins.
> >>
> >> Reflecting on our earlier conversation(s), I think I can see why you're
> >> asking this. :-) The augmented type information that'll be needed to do
> >> this completely/properly will actually have to associate types with
> field
> >> paths (not just with fields by name) - which is a slightly more
> complicated
> >> association.
> >>
> >> Cheers,
> >> Mike
> >>
> >>
> >> On 7/13/17 10:54 PM, Yingyi Bu wrote:
> >>
> >>> Hi Taewoo,
> >>>
> >>> The first query shouldn't fail because indexnl is just a hint.
> >>> The second query should succeed because it's a valid indexing
> statement.
> >>> High nesting levels in open record like JSON is not uncommon.
> >>>
> >>> Best,
> >>> Yingyi
> >>>
> >>>
> >>> On Thu, Jul 13, 2017 at 10:51 PM, Taewoo Kim <wangsaeu@gmail.com>
> wrote:
> >>>
> >>> @Mike: In order to properly deal with the enforced index on a
> nested-type
> >>>> field, I need to make sure that whether my understanding (each nested
> >>>> type
> >>>> (except the leaf level0 has a record type for the next level) is
> correct
> >>>> or
> >>>> not. Which one is a bug? The first one (without index) should fail?
Or
> >>>> the
> >>>> second one (with an index) should succeed?
> >>>>
> >>>> Best,
> >>>> Taewoo
> >>>>
> >>>> On Thu, Jul 13, 2017 at 9:58 PM, Yingyi Bu <buyingyi@gmail.com>
> wrote:
> >>>>
> >>>> Indeed, it's a bug!
> >>>>>
> >>>>> Best,
> >>>>> Yingyi
> >>>>>
> >>>>> On Thu, Jul 13, 2017 at 9:52 PM, Mike Carey <dtabass@gmail.com>
> wrote:
> >>>>>
> >>>>> Sounds like a bug to me.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On 7/13/17 7:59 PM, Taewoo Kim wrote:
> >>>>>>
> >>>>>> Currently, I am working on a field type propagation without
using
> >>>>>>> initializing the OptimizableSubTree in the current index
access
> >>>>>>>
> >>>>>> method.
> >>>>
> >>>>> I
> >>>>>
> >>>>>> am encountering an issue with an open-type enforced index. So,
I
> just
> >>>>>>>
> >>>>>> want
> >>>>>
> >>>>>> to make sure that my understanding is correct. It looks like
we
> can't
> >>>>>>>
> >>>>>> have
> >>>>>
> >>>>>> an enforced-index on a completely schemaless nested field. For
> >>>>>>>
> >>>>>> example,
> >>>>
> >>>>> the
> >>>>>>> following doesn't generate any issue.
> >>>>>>>
> >>>>>>> //
> >>>>>>> create type DBLPType as open {id: int32}
> >>>>>>> create type CSXType as closed {id: int32}
> >>>>>>>
> >>>>>>> create dataset DBLP(DBLPType) primary key id;
> >>>>>>> create dataset CSX(CSXType) primary key id;
> >>>>>>>
> >>>>>>> for $a in dataset('DBLP')
> >>>>>>> for $b in dataset('CSX')
> >>>>>>> where $a.nested.one.title /*+ indexnl */ = $b.nested.one.title
> >>>>>>> return {"arec": $a, "brec": $b}
> >>>>>>> //
> >>>>>>>
> >>>>>>> However, the following generates an exception. So, can we
assume
> that
> >>>>>>>
> >>>>>> to
> >>>>
> >>>>> create an enforced-index, except the leaf level, there should be
a
> >>>>>>>
> >>>>>> defined
> >>>>>
> >>>>>> record type. For example, for this example, there should be
"nested"
> >>>>>>>
> >>>>>> type
> >>>>>
> >>>>>> and "one" type.
> >>>>>>>
> >>>>>>> //
> >>>>>>> create type DBLPType as open {id: int32}
> >>>>>>> create type CSXType as closed {id: int32}
> >>>>>>>
> >>>>>>> create dataset DBLP(DBLPType) primary key id;
> >>>>>>> create dataset CSX(CSXType) primary key id;
> >>>>>>>
> >>>>>>> create index title_index_DBLP on DBLP(nested.one.title:
string?)
> >>>>>>>
> >>>>>> enforced;
> >>>>>
> >>>>>> create index title_index_CSX on CSX(nested.one.title: string?)
> >>>>>>>
> >>>>>> enforced;
> >>>>
> >>>>> for $a in dataset('DBLP')
> >>>>>>> for $b in dataset('CSX')
> >>>>>>> where $a.nested.one.title /*+ indexnl */ = $b.nested.one.title
> >>>>>>> return {"arec": $a, "brec": $b}
> >>>>>>> //
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Taewoo
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>
>
> Best regards,
> Ildar
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message