hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: trying to write output to a file whose name i can have control over
Date Thu, 10 Jun 2010 16:27:06 GMT
A better way (than holding a static variable) would be to define a
Configuration key-value and use that from within your method to set
the name each time. The Context object can help you retrieve the set
configuration value.

On Thu, Jun 10, 2010 at 6:19 PM, Giridhar Addepalli
<giridhar.addepalli@komli.com> wrote:
> Hi,
>
>
>
> I am using hadoop 0.20.2
>
> Maperduce framework by default writes output to part-r-0000 etc.
>
> I want to write to a file with different name.
>
>
>
> I am trying to override “getDefaultWorkFile” method in TextOutputFormat
> class.
>
> I am getting following error :
>
>
>
> java.lang.IllegalArgumentException: Can not create a Path from a null string
>
>                 at org.apache.hadoop.fs.Path.checkPathArg(Path.java:78)
>
>                 at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>
>                 at org.apache.hadoop.fs.Path.<init>(Path.java:50)
>
>                 at
> com.output.CustomizeOutputFormat.getDefaultWorkFile(CustomizeOutputFormat.java:38)
>
>                 at
> org.apache.hadoop.mapreduce.lib.output.TextOutputFormat.getRecordWriter(TextOutputFormat.java:125)
>
>                 at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
>
>                 at
> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>
>                 at org.apache.hadoop.mapred.Child.main(Child.java:170)
>
>
>
>
>
> Please find attached source files.
>
>
>
> Please help,
>
> Giridhar.



-- 
Harsh J
www.harshj.com

Mime
View raw message