hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mapred Learn <mapred.le...@gmail.com>
Subject Fwd: How to create Output files of about fixed size
Date Wed, 26 Oct 2011 14:03:33 GMT

> 

> Hi,
> I am trying to create output files of fixed size by using : 
> -Dmapred.max.split.size=6442450812 (6 Gb)
> 
> But the problem is that the input Data size and metadata varies  and I have to adjust
above value manually to achieve fixed size.
> 
> Is there a way I can programmatically determine split size that would yield me fixed
sized output files. For eg 200 MB each ?
> 
> Thanks,
> JJ

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message