hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From steven zhuang <zhuangxin8...@gmail.com>
Subject ReflectionUtils.setConf would configure anything Configurable twice?
Date Fri, 24 Jun 2011 02:22:37 GMT
hello, list,

              Recently I have upgraded our Hadoop cluster from 0.20.2 to
0.21.0, and I found there is something wrong introduced by the upgrade.

              In the setConf method of
org.apache.hadoop.util.ReflectionUtils, any instance of Configurable would
be configured twice, and this may cause trouble.
              For example, in 0.21.0, KeyFieldBasedPartitioner implements
the Configurable interface. When configured twice, it get two KeyDescription
and gives out wrong partition number.

I have created a ticket for this:

Paste the source code in 0.21.0 (and 0.20.2 too) below:

public static void setConf(Object theObject, Configuration conf) {
    if (conf != null) {
       if (theObject instanceof Configurable) {
           ((Configurable) theObject).setConf(conf);
      setJobConf(theObject, conf);

        best wishes.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message