hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: How to set SequenceFile.Metadata from within SequenceFileOutputFormat?
Date Mon, 09 Aug 2010 21:45:28 GMT
On 08/09/2010 04:01 PM, David Rosenstrauch wrote:
> On a similar note, it looks like if I want to customize the name/path of
> the generated SequenceFile my only option currently is to override
> FileOutputFormat.getDefaultWorkFile().
> a) Again, have I got this correct, or am I overlooking something?
> b) Would anyone else agree that this is something that can/should be
> made easier? (And thus worthy of a bug report?)
> Thanks,
> DR

Ugh.  Actually, this looks even worse than I thought.

It looks like there's a bunch of static helper methods in 
FileOutputFormat which use methods other than getDefaultWorkFile() to 
determine the file name.

It looks like most of them use the method getUniqueFile().  Problem is 
that getUniqueFile is a *static* method, so I can't override it with an 
alternate implementation.

Anyone know any short way out of this conundrum without my having to 
completely reimplement chunks of FileOutputFormat/StateFileOutputFormat?



View raw message