hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Answer Agrawal <yrsna.tse...@gmail.com>
Subject Re: How to set mapreduce.input.fileinputformat.split.maxsize for a specific job
Date Sun, 17 May 2015 14:17:15 GMT
Thanks,
Is this the correct way to write ?
conf.set("mapreduce.input.fileinputformat.split.maxsize", "102400");
or
job.getConfiguration().set("mapreduce.input.fileinputformat.split.maxsize",
"102400");

I think another ways as
FileInputFormat.setMaxInputSplitSize(null, 102400);

Is this all right ? Are these both solve the same purpose or something else
?

Thanks,

On Sat, May 16, 2015 at 8:48 PM, Shahab Yunus <shahab.yunus@gmail.com>
wrote:

> You can either pass them on as command line argument using -D option.
> Assuming your job is implementing the standard Tool interface:
> https://hadoop.apache.org/docs/current/api/org/apache/hadoop/util/Tool.html
>
> Or you can set them in the code using the various 'set' methods to set
> key/value values in the configuration object.
>
> ...
> Job job = Job.getInstance(getConf());
> job.setJarByClass(MyJob.class);
>
> job.getConfiguration().set("<property-name>",<value>);
> ....
>
> Docs for Configuration class:
> https://hadoop.apache.org/docs/current/api/org/apache/hadoop/conf/Configuration.html
>
> This will work as long as the property is not marked final
>
> Regards,
> Shahab
>
>
> On Sat, May 16, 2015 at 10:49 AM, Answer Agrawal <yrsna.tset01@gmail.com>
> wrote:
>
>> Hi,
>>
>> In xmls configuration file of Hadoop-2.x,
>> "mapreduce.input.fileinputformat.split.minsize" is given which can be set
>> but how to set "mapreduce.input.fileinputformat.split.maxsize" in xml file.
>> I need to set it in my mapreduce code.
>>
>> Thanks,
>>
>
>

Mime
View raw message