hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai" <dai...@gmail.com>
Subject Re: Is it possible to access Configuration in UDF ?
Date Tue, 04 Aug 2009 04:19:48 GMT
Hi, Jeff,
This is not API at all, this is a hack to make things work. We do lack 
couples of features for UDF:
1. reporter and counter (PIG-889)
2. access global properties
3. ability to maintain states across different UDF invocations
4. input schema
5. variable length arguments (PIG-902)

Your suggestion sounds resonable. We need to provide a well designed 
interface for these features.

----- Original Message ----- 
From: "zhang jianfeng" <zjffdu@gmail.com>
To: <pig-user@hadoop.apache.org>; <pig-dev@hadoop.apache.org>
Sent: Monday, August 03, 2009 8:03 PM
Subject: Re: Is it possible to access Configuration in UDF ?


> Dmitriy,
>
> Thank you for your help.
>
> I find this way of using API is not so intuitive ,  I recommend the base
> class of UDF to implements the Configurable interface.
> Then each UDF can use the getConf() to get the Configuration object.
> Because UDF is part of MapReduce , it makes sense to make it Configurable.
>
> The following is what I recommend to change the EvalFunc
>
> public abstract class EvalFunc<T>  implements Configurable{
> ......
> protected Configuration conf;
> ......
> public EvalFunc(){
>         conf=PigMapReduce.sJobConf;
> }
> ......
>     @Override
>    public void setConf(Configuration conf) {
>        this.conf=conf;
>    }
>
>    @Override
>    public Configuration getConf() {
>        return this.conf;
>    }
>
>
>
>
> Jeff Zhang
>
>
>
>
>
> On Mon, Aug 3, 2009 at 8:52 PM, Dmitriy Ryaboy 
> <dvryaboy@cloudera.com>wrote:
>
>> You can access the JobConf with the following call:
>>
>> ConfigurationUtil.toProperties(PigMapReduce.sJobConf)
>>
>> On Mon, Aug 3, 2009 at 12:40 AM, zhang jianfeng<zjffdu@gmail.com> wrote:
>> > Hi all,
>> >
>> > I'd like to set property in Configuration to customize my UDF. But  it
>> looks
>> > like I can not access the Configuration object in UDF.
>> >
>> > Does pig have a plan to support this feature ?
>> >
>> >
>> > Thank you.
>> >
>> > Jeff Zhang
>> >
>>
> 


Mime
View raw message