hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Parth J. Brahmbhatt" <pa...@ccs.neu.edu>
Subject Problem while writing a file from Reducer
Date Sat, 05 Dec 2009 00:15:18 GMT
Hello All,

I am creating 2 files in the constructor of my reducer and I store the file Handles as member
variables. I write some data in these files for each call to reduce method. For some reason
the files are created but only the larger file has data(The larger file is 400mb).The smaller
file is created but no data is written to it (The size should be around 4MB). I checked the
hadoop configuration and the buffer-size parameter is 4096. 

I padded some 100MB data to the smaller file and than this file gets around 64MB data. I am
clueless what is happening and if there is a solution to this. Please share if you have faced
similar problem or if you know the solution.

Parth Brahmbhatt

View raw message