hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pankit Thapar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-7201) Fix TestHiveConf#testConfProperties test case
Date Mon, 09 Jun 2014 22:23:05 GMT
Pankit Thapar created HIVE-7201:
-----------------------------------

             Summary: Fix TestHiveConf#testConfProperties test case
                 Key: HIVE-7201
                 URL: https://issues.apache.org/jira/browse/HIVE-7201
             Project: Hive
          Issue Type: Bug
          Components: Tests
    Affects Versions: 0.13.0
            Reporter: Pankit Thapar
            Priority: Minor


CHANGE 1: 

TEST CASE :
The intention of TestHiveConf#testConfProperties() is to test the HiveConf properties being
set in the priority as expected.

Each HiveConf object is initialized as follows:
1) Hadoop configuration properties are applied.
2) ConfVar properties with non-null values are overlayed.
3) hive-site.xml properties are overlayed.

ISSUE :
The mapreduce related configurations are loaded by JobConf and not Configuration.
The current test tries to get the configuration properties  like : HADOOPNUMREDUCERS ("mapred.job.reduces")
from Configuration class. But these mapreduce related properties are loaded by JobConf class
from mapred-default.xml.

DETAILS :
LINE  63 : checkHadoopConf(ConfVars.HADOOPNUMREDUCERS.varname, "1"); -->fails
Because, 
private void  checkHadoopConf(String name, String expectedHadoopVal) {
     Assert.assertEquals(expectedHadoopVal, new Configuration().get(name)); ----> Second
parameter is null, since its the JobConf class and not the Configuration class that initializes
mapred-default values. 
}

Code that loads mapreduce resources is in ConfigUtil and JobConf makes a call like this (in
static block):
public class JobConf extends Configuration {
  
  private static final Log LOG = LogFactory.getLog(JobConf.class);

  static{
    ConfigUtil.loadResources(); --> loads mapreduce related resources (mapreduce-default.xml)
  }
.....
}

Please note, the test case assertion works fine if HiveConf() constructor is called before
this assertion since, HiveConf() triggers JobConf()
which basically sets the default values of the properties pertaining to mapreduce.
This is why, there won't be any failures if testHiveSitePath() was run before testConfProperties()
as that would load mapreduce
properties into config properties.


FIX:
Instead of using a Configuration object, we can use the JobConf object to get the default
values used by hadoop/mapreduce.

CHANGE 2:
In TestHiveConf#testHiveSitePath(), a call to static method getHiveSiteLocation() should be
called statically instead of using an object.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message