hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Initialization costs
Date Mon, 30 Oct 2006 13:47:22 GMT
I know in general that I shouldn't worry too much about  
initialization costs, as they will be amortized over the life of the  
job and are often a drop in the bucket time wise.  However, in my  
setup I have a conf() method that needs to load in some resources  
from disk.   This is on a per job basis currently.  I know that each  
node in my cluster is going to need these resources and every job I  
submit is going to end up doing this same thing.  So I was wondering  
if there was anyway these resources could be loaded once per startup  
of the task tracker.  In some sense, this is akin to putting  
something into application scope in a webapp as opposed to session  


View raw message