drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jinfeng Ni <jinfengn...@gmail.com>
Subject Re: Classpath scanning & udfs
Date Mon, 11 Jan 2016 18:28:31 GMT
If what Rahul described is true, I think at minimum we should document
the backward compatible issue in Drill new release doc. For today,
drill doc seems to still use the old way [1]

[1] https://drill.apache.org/docs/adding-custom-functions-to-drill/


On Mon, Jan 11, 2016 at 10:17 AM, rahul challapalli
<challapallirahul@gmail.com> wrote:
> Thanks for your reply Jason.
>
> If we cannot override the global configuration file, then for existing
> UDF's we have to re-compile them by modifying the drill-module.conf file.
> If so our UDF's are not backward compatible. Appreciate it if someone can
> confirm this.
>
> - Rahul
>
> On Mon, Jan 11, 2016 at 9:59 AM, Jason Altekruse <altekrusejason@gmail.com>
> wrote:
>
>> Rahul,
>>
>> The error message you are seeing is in reading a storage plugin
>> configuration file. I am planning to fix these kinds of messages to
>> actually direct users at the file that is failing parsing. I have seen this
>> in the past when the classpath was incorrect and one of the plugins (like
>> Hbase) was not included.
>>
>> Julien can confirm, but I think this might be intentional to have the paths
>> read out of the modules configuration rather than the global one to save
>> time when scanning the path (rather than scanning all of the jars for all
>> paths given in the override file).
>>
>> On Fri, Jan 8, 2016 at 4:32 PM, rahul challapalli <
>> challapallirahul@gmail.com> wrote:
>>
>> > Before 1.2, my udfs project contained an empty drill-override.conf file
>> and
>> > I used to update the drill-override.conf on all the drillbits to specify
>> > the package of my UDF. This is no longer working for me. I tried a few
>> > things and below is how my drill-override.conf file looks now
>> >
>> > drill.classpath.scanning.packages :
>> ${?drill.classpath.scanning.packages} [
>> > org.apache.drill.udfs ]
>> > drill.exec: {
>> >   cluster-id: "rahul_cluster_com-drillbits",
>> >   zk.connect: "localhost:5181"
>> > }
>> >
>> > When I restart the drillbits, I get this strange error " Caused by:
>> > com.fasterxml.jackson.databind.JsonMappingException: Could not resolve
>> type
>> > id 'hbase' into a subtype of [simple type, class
>> > org.apache.drill.common.logical.StoragePluginConfig]"
>> >
>> > If I moved the package information to the drill-module.conf in my udf's
>> > project, then things are working fine. However this requires re-compiling
>> > the udfs which is not desirable. Is there any other way around this ?
>> >
>> > - Rahul
>> >
>>

Mime
View raw message