hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathan Marz <nat...@rapleaf.com>
Subject Custom input format getSplits being called twice
Date Thu, 25 Sep 2008 17:49:23 GMT
Hello all,

I am getting some odd behavior from hadoop which seems like a bug. I  
have created a custom input format, and I am observing that my  
"getSplits" method is being called twice. Each call is on a different  
instance of the input format. The job, however, is only run once,  
using the result from the second call to getSplits. The first call  
receives the numSplits hint as expected, while in the second call that  
value is overriden to 1. I am running hadoop in standalone mode. Does  
anyone know anything about this issue?

Thanks,

Nathan Marz
Rapleaf

Mime
View raw message