hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venu Gopala Rao (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-7239) The variables in the configuration files are not replaced with values, when the job is submitted by Job client
Date Fri, 22 Apr 2011 18:01:06 GMT
The variables in the configuration files are not replaced with values, when the job is submitted
by Job client
--------------------------------------------------------------------------------------------------------------

                 Key: HADOOP-7239
                 URL: https://issues.apache.org/jira/browse/HADOOP-7239
             Project: Hadoop Common
          Issue Type: Bug
          Components: conf
    Affects Versions: 0.21.0, 0.20.2
         Environment: Linux + Sun JDK 1.6
            Reporter: Venu Gopala Rao


We have a case where we wanted to create the Job names dynamically at run time.Since JobConf
is an extension for the Configuration object, we thought we can make use of the variable substitution
concept in configuration like below

Job job = new Job(conf, "${mapred.user.name}" + "-job" + new Random().nextInt()); 
job.setJarByClass(WordCount.class); 
job.setMapperClass(TokenizerMapper.class); 
job.setCombinerClass(IntSumReducer.class); 
job.setReducerClass(IntSumReducer.class); 
job.setOutputKeyClass(Text.class); 
job.setOutputValueClass(IntWritable.class); 
FileInputFormat.addInputPath(job, new Path(otherArgs[0])); 
FileOutputFormat.setOutputPath(job, new Path(otherArgs[1])); 
job.submit(); 


We set the required run time variables(in this case mapred.user.name) before calling the job
submit. But on the Job tracker side the variables are not replaced correctly.


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message