hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tianyin Xu <t...@cs.ucsd.edu>
Subject Re: Are these configuration parameters deprecated?
Date Fri, 14 Nov 2014 21:54:52 GMT
So, these parameters are indeed deprecated?

Then how do I find the replaced one? Let's just say
"mapreduce.task.tmp.dir," how which one should I set to have the same
behavior?

@Niels, yes, exactly, Hadoop in particular has a high churn of
configuration renaming which is pretty annoying. Not even the
deprecated parameters are not complained, but also any invalid
parameters are like this.

~t


On Fri, Nov 14, 2014 at 1:03 PM, Niels Basjes <Niels@basjes.nl> wrote:
> A while ago I found a similar problem;
> I was wondering why do a lot of tools like the hdfs, hbase shell, pig and
> many other complain at startup about deprecated parameters.
> It turns out that these deprecated names are still in *-default.xml files
> and in various other places in the code base.
>
> Perhaps an issue indicating that the use of the deprecated parameters should
> be removed from the main code base is in order here.
>
> Niels Basjes
>
> On Fri, Nov 14, 2014 at 9:22 PM, Tianyin Xu <tixu@cs.ucsd.edu> wrote:
>>
>> Hi,
>>
>> I'm very confused by some of the MapReduce configuration parameters
>> which appear in the latest version of mapred-default.xml.
>>
>> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
>>
>> Take "mapreduce.task.tmp.dir" as an example, I fail to find its usage
>> in code but
>>
>> /* mapreduce/util/ConfigUtil.java */
>>  55       new DeprecationDelta("mapred.temp.dir",
>>  56         MRConfig.TEMP_DIR),
>>
>> My interpretation is that it's renamed into "mapred.temp.dir".
>> However, when I grep the new name, I still cannot find any code except
>> some testing ones in
>>
>> ./hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BenchmarkThroughput.java
>>
>> From the semantics, this should be a "must-to-have" parameter for MR
>> jobs...
>>
>> Also, many parameters are like this. So I'm really confused.
>>
>> Am I missing something? Are these parameters deprecated?
>>
>> Thanks a lot!
>> Tianyin
>
>
>
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes



-- 
Tianyin XU,
http://cseweb.ucsd.edu/~tixu/

Mime
View raw message