Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Wed, 13 Jul 2011 11:32:00 +0000 (UTC)
From: "Sudharsan Sampath (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: 
 <1956770189.9518.1310556720128.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1304349125.35232.1308881147698.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HADOOP-7425) ReflectionUtils.setConf would
 configure anything Configurable twice
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-7425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064494#comment-13064494 ] 

Sudharsan Sampath commented on HADOOP-7425:
-------------------------------------------

Hi Steven,

Yes. You are right. This issue is seen on the trunk too. But to me, configuring twice should not be an issue but processing the configured property without realizing that the same configuration has been processed already could be avoided.

Will it be ok if instead of maintaining a List, we maintain a Set of keyspecs in the KeyFieldHelper and provide an equals method that returns true if its params match exactly as the passed in. The downside is that if some job wants this kind of double hashing done on the same keyspec that would be difficult asking for a custom partitioner. But I doubt if there would be such a scenario.

This would also ensure we get the correct partition and increase performance as we avoid a hash invoke for every record. 

> ReflectionUtils.setConf would configure anything Configurable twice
> -------------------------------------------------------------------
>
>                 Key: HADOOP-7425
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7425
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: util
>    Affects Versions: 0.21.0, 0.23.0
>            Reporter: steven zhuang
>         Attachments: test.tar
>
>
> In  the setConf method of org.apache.hadoop.util.ReflectionUtils, any instance of Configurable would be configured twice.
> In 0.21.0, KeyFieldBasedPartitioner implements the Configurable interface. When configured twice, it get two KeyDescription and gives out wrong partition number. 
> public static void setConf(Object theObject, Configuration conf) {
>     if (conf != null) {
>       if (theObject instanceof Configurable) {
>         ((Configurable) theObject).setConf(conf);
>       }
>       setJobConf(theObject, conf);
>     }
>   }

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira