Mailing-List: contact hadoop-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-user@lucene.apache.org
Received-SPF: neutral (herse.apache.org: local policy)
Mime-Version: 1.0 (Apple Message framework v752.2)
Content-Transfer-Encoding: 7bit
Message-Id: <5CFC98AE-894B-4146-B76D-5B6EE13B2044@apache.org>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
To: hadoop-user@lucene.apache.org
From: Grant Ingersoll <gsingers@apache.org>
Subject: Initialization costs
Date: Mon, 30 Oct 2006 08:47:22 -0500

I know in general that I shouldn't worry too much about  
initialization costs, as they will be amortized over the life of the  
job and are often a drop in the bucket time wise.  However, in my  
setup I have a conf() method that needs to load in some resources  
from disk.   This is on a per job basis currently.  I know that each  
node in my cluster is going to need these resources and every job I  
submit is going to end up doing this same thing.  So I was wondering  
if there was anyway these resources could be loaded once per startup  
of the task tracker.  In some sense, this is akin to putting  
something into application scope in a webapp as opposed to session  
scope.

Thanks,
Grant