hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarus, Nathan" <jar...@amazon.com>
Subject DBOutputWriter timing out writing to database
Date Thu, 02 Aug 2012 19:04:52 GMT
Hey,

I'm running Hadoop 0.20.205 and am using the DBOutputFormat to write to a database. For small
datasets, my jobs work perfectly, but for larger jobs, writing to the database takes longer
than 600 seconds and Hadoop times out my reduce tasks. Looking at the source for DBOutputFormat,
it seems the Progressable never gets updated while the insert query is being run. How do I
modify/subclass DBOutputFormat to update this so my jobs can finish?

Thanks
Nathan
Mime
View raw message