hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Realtime Map Reduce = Supercomputing for the Masses?
Date Mon, 02 Jun 2008 10:23:27 GMT
Christophe Taton wrote:
> Actually Hadoop could be made more friendly to such realtime Map/Reduce
> jobs.
> For instance, we could consider running all tasks inside the task tracker
> jvm as separate threads, which could be implemented as another personality
> of the TaskRunner.
> I have been looking into this a couple of weeks ago...
> Would you be interested in such a feature?

Why does that have benefits? So that you can share stuff via local data 
structures? Because you'd better be sharing classloaders if you are 
going to play that game. And that is very hard to get right (to the 
extent that I dont think any apache project other than Felix does it well)

View raw message