drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdel Hakim Deneche <adene...@maprtech.com>
Subject Re: UDF is not recognized by drill / Validation Error
Date Fri, 20 May 2016 15:30:13 GMT
Julian,

can you share the both jars (classes and sources) with us. I will give it a
try and see if I can figure out what's going on.

Thanks

On Fri, May 20, 2016 at 6:08 AM, Julian Feinauer <julian.feinauer@web.de>
wrote:

> Dear all,
>
> thank you very much for all your replies.
> I tried everything but it is still not working.
>
> 1. I copy both files (classes and sources) in the /jars/3rdparty directory
> 2. I restarted the drillbit after this (I use only one drillbit and
> drill-conf both running on my local machine)
> 3. I changed the class to a static subclass
> 4. I have the drill-module.conf in my ressources
> 4. The error appears in the drillbit.log as soon as i call the udf because
> it is not recognized by drill.
> On startup drillbit.log states all the packages and jars that are scanned
> and my custom jar is not listed there.
> Therefore I think it is a problem with the class loader or something
> related?
>
> Could this be possible?
>
> Greetings
> Julian
>
> > Am 20.05.2016 um 14:36 schrieb Tugdual Grall <tugdual@gmail.com>:
> >
> > Hi
> >
> > Be sure you deploy on each nodes, 2 jars:
> > - the jar containing the classes
> > - the jar contains the sources
> >
> > The POM.xml in the simple function examples contains the maven
> > configuration to generate these 2 files, be sure you have the same in
> your
> > project:
> >
> https://github.com/tgrall/drill-simple-mask-function/blob/master/pom.xml#L24-L37
> >
> > and you have restarted the drillbit
> >
> > Regards
> > Tug
> > @tgrall
> >
> >
> >
> > On Fri, May 20, 2016 at 12:34 AM, Abdel Hakim Deneche <
> adeneche@maprtech.com
> >> wrote:
> >
> >> the example I gave you was incomplete, here is what I meant to send:
> >>
> >> public class MyUDF {
> >>
> >>   @FunctionTemplate(name = „myaddints", scope = FunctionTemplate.
> >>   FunctionScope.SIMPLE, nulls =
> >> FunctionTemplate.NullHandling.NULL_IF_NULL)
> >>   public *static *class IntIntAdd implements DrillSimpleFunc {
> >>      ...
> >>   }
> >>
> >> }
> >>
> >>
> >> On Thu, May 19, 2016 at 3:33 PM, Abdel Hakim Deneche <
> >> adeneche@maprtech.com>
> >> wrote:
> >>
> >>> Hey Julian,
> >>>
> >>> one more thing you could try out: declare the UDF as a static class
> >> inside
> >>> another class:
> >>>
> >>> public class MyUDF {
> >>>
> >>>   @FunctionTemplate(name = „myaddints", scope = FunctionTemplate.
> >>>   FunctionScope.SIMPLE, nulls = FunctionTemplate.NullHandling.
> >>> NULL_IF_NULL)
> >>>   public class IntIntAdd implements DrillSimpleFunc {
> >>>      ...
> >>>   }
> >>>
> >>> }
> >>>
> >>> Take a look at the following page to see an examples of UDFs:
> >>> http://drill.apache.org/docs/custom-function-interfaces/
> >>>
> >>> If this doesn't work check the drillbit log, it should print an error
> >>> message when it's starting up if something's wrong with your UDF.
> >>>
> >>> Thanks
> >>>
> >>>
> >>> On Thu, May 19, 2016 at 3:31 AM, Julian Feinauer <
> julian.feinauer@web.de
> >>>
> >>> wrote:
> >>>
> >>>> Dear folks,
> >>>>
> >>>> I’m currently experimenting with user defined functions in drill but
> I’m
> >>>> not able to get them to work on my drillbits.
> >>>> I always get the error: Error: VALIDATION ERROR: From line 1, column
8
> >> to
> >>>> line 1, column 41: No match found for function signature
> >> myaddints(<ANY>,
> >>>> <ANY>).
> >>>>
> >>>> I already went through all the tips I found in the mailing list.
> >>>> The jar contains a drill-module.conf with the content:
> >>>> drill.classpath.scanning.packages += "org.julian"
> >>>> And the UDF is defined as:
> >>>> package org.julian;
> >>>>
> >>>> import ...
> >>>>
> >>>> @FunctionTemplate(name = „myaddints", scope =
> >>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> >>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> >>>> public class IntIntAdd implements DrillSimpleFunc {
> >>>>
> >>>>    @Param
> >>>>    IntHolder in1;
> >>>>
> >>>>    @Param
> >>>>    IntHolder in2;
> >>>>
> >>>>    @Output
> >>>>    IntHolder out;
> >>>>
> >>>>    public void setup() {};
> >>>>
> >>>>    public void eval() {
> >>>>        out.value = in1.value + in2.value;
> >>>>    }
> >>>>
> >>>> }
> >>>> I tried to find the bug using the logs and encountered the following
> >>>> messages from the drillbit.log:
> >>>> Base Configuration:
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-common-1.6.0.jar!/drill-default.conf
> >>>>
> >>>> Intermediate Configuration and Plugin files, in order of precedence:
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-gis-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-logical-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-kudu-storage-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/3rdparty/drill-memory-base-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-hive-exec-shaded-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-memory-base-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-mongo-storage-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-java-exec-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-common-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-storage-hbase-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-jdbc-storage-1.6.0.jar!/drill-module.conf
> >>>>        -
> >>>>
> >>
> jar:file:/usr/local/Cellar/apache-drill/1.6.0/libexec/jars/drill-storage-hive-core-1.6.0.jar!/drill-module.conf
> >>>>
> >>>> which let’s me think that my jar file (drillUDF-1.0-SNAPSHOT.jar)
is
> not
> >>>> scanned altough it is located in jars/3rdparty.
> >>>>
> >>>> Am I doing something wrong or am I missing something or has anyone a
> >> good
> >>>> hint for me?
> >>>>
> >>>> Thank you already!
> >>>> Julian
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>>
> >>> Abdelhakim Deneche
> >>>
> >>> Software Engineer
> >>>
> >>>  <http://www.mapr.com/>
> >>>
> >>>
> >>> Now Available - Free Hadoop On-Demand Training
> >>> <
> >>
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >>
> >> Abdelhakim Deneche
> >>
> >> Software Engineer
> >>
> >>  <http://www.mapr.com/>
> >>
> >>
> >> Now Available - Free Hadoop On-Demand Training
> >> <
> >>
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >>>
> >>
>
>


-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message