drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Re: UDF to fetch fields from a Lucene Index - Does something similar exist
Date Fri, 29 Jan 2016 21:11:04 GMT
I think maybe Ted Dunning built something against HBase leveraging this
technique. Ted, am I remembering right and if so, is that code somewhere
public?

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Fri, Jan 29, 2016 at 12:58 PM, Jason Altekruse <altekrusejason@gmail.com>
wrote:

> For a function that uses complex writer you could look at the
> convert_fromJSON [1] function, which takes a varchar containing JSON and
> parses it into a complex structure.
>
> The best example of reading from an external source would be the record
> readers for the actual format/storage plugins. There aren't really many
> limitations to what you can do from a UDF, so you can just put the
> initialization of your reader on the Lucene index in the setup method of
> the function and then read out if it as you would from a within a record
> reader.
>
> The JSON functions might be particularly useful, because it actually shares
> the reader code between the record reader and the UDF, so it should be able
> to demonstrate reasonable structure for sharing the code between the lucene
> reader and the function.
>
> [1] -
>
> https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/JsonConvertFrom.java
>
> On Fri, Jan 29, 2016 at 11:56 AM, Stefán Baxter <stefan@activitystream.com
> >
> wrote:
>
> > Hi,
> >
> > We want to create and share a UDF that fetches additional query fields
> from
> > a Lucene index (via Complex writer).
> >
> > I'm wondering if anything similar existed for Redis, Hbase or any other
> > data source. It would be greatly appreciated if someone could point me
> to a
> > meaningful starting point.
> >
> > Regards,
> >  -Stefan
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message