drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefán Baxter <ste...@activitystream.com>
Subject Re: UDF to fetch fields from a Lucene Index - Does something similar exist
Date Sun, 31 Jan 2016 14:27:11 GMT
Any takers?

thanks for the pointers.

-Stefan

On Fri, Jan 29, 2016 at 9:11 PM, Jacques Nadeau <jacques@dremio.com> wrote:

> I think maybe Ted Dunning built something against HBase leveraging this
> technique. Ted, am I remembering right and if so, is that code somewhere
> public?
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Fri, Jan 29, 2016 at 12:58 PM, Jason Altekruse <
> altekrusejason@gmail.com>
> wrote:
>
> > For a function that uses complex writer you could look at the
> > convert_fromJSON [1] function, which takes a varchar containing JSON and
> > parses it into a complex structure.
> >
> > The best example of reading from an external source would be the record
> > readers for the actual format/storage plugins. There aren't really many
> > limitations to what you can do from a UDF, so you can just put the
> > initialization of your reader on the Lucene index in the setup method of
> > the function and then read out if it as you would from a within a record
> > reader.
> >
> > The JSON functions might be particularly useful, because it actually
> shares
> > the reader code between the record reader and the UDF, so it should be
> able
> > to demonstrate reasonable structure for sharing the code between the
> lucene
> > reader and the function.
> >
> > [1] -
> >
> >
> https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/JsonConvertFrom.java
> >
> > On Fri, Jan 29, 2016 at 11:56 AM, Stefán Baxter <
> stefan@activitystream.com
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > We want to create and share a UDF that fetches additional query fields
> > from
> > > a Lucene index (via Complex writer).
> > >
> > > I'm wondering if anything similar existed for Redis, Hbase or any other
> > > data source. It would be greatly appreciated if someone could point me
> > to a
> > > meaningful starting point.
> > >
> > > Regards,
> > >  -Stefan
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message