drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Altekruse <altekruseja...@gmail.com>
Subject Re: UDF to fetch fields from a Lucene Index - Does something similar exist
Date Fri, 29 Jan 2016 20:58:54 GMT
For a function that uses complex writer you could look at the
convert_fromJSON [1] function, which takes a varchar containing JSON and
parses it into a complex structure.

The best example of reading from an external source would be the record
readers for the actual format/storage plugins. There aren't really many
limitations to what you can do from a UDF, so you can just put the
initialization of your reader on the Lucene index in the setup method of
the function and then read out if it as you would from a within a record
reader.

The JSON functions might be particularly useful, because it actually shares
the reader code between the record reader and the UDF, so it should be able
to demonstrate reasonable structure for sharing the code between the lucene
reader and the function.

[1] -
https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/JsonConvertFrom.java

On Fri, Jan 29, 2016 at 11:56 AM, Stefán Baxter <stefan@activitystream.com>
wrote:

> Hi,
>
> We want to create and share a UDF that fetches additional query fields from
> a Lucene index (via Complex writer).
>
> I'm wondering if anything similar existed for Redis, Hbase or any other
> data source. It would be greatly appreciated if someone could point me to a
> meaningful starting point.
>
> Regards,
>  -Stefan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message