hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Best practice for in memory data?
Date Thu, 25 Jan 2007 17:28:14 GMT
Johan Oskarsson wrote:
> Any advice on how to solve this problem?

I think your current solutions sound reasonable.

> Would it be possible to somehow share a hashmap between tasks?

Not without running multiple tasks in the same JVM.  We could implement 
a mode where child tasks are run directly in the JobTracker's JVM, but 
that would not be good for reliability.  Alternately we could have 
spawned child processes execute multiple tasks, perhaps even in 
parallel, extending the TaskUmbilicalProtocol(), JobTracker, etc.  This 
would unfortunately not be a trivial modification.


View raw message