hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Groschupf ...@media-style.com>
Subject find a tasktracker that process a named task
Date Fri, 17 Feb 2006 00:06:57 GMT
this is in my mind since some time.
Since the global locking discussion in the nutch list, I would love  
to inject my thoughts into this list.

First let me give a use case:
We have some nutch index search servers running as MapRunnable tasks.
The problem: how a client that want's to connect to this search  
servers knows all the hosts that runs this runnable and may track  

Once I had implement a kind of prototype for this use case and done  
it like this:
I had start a rpc server on the search server client side (my tomcat  
host) and all search servers was registering itself there. Than the  
client was using this repository to lookup the search servers.

I think there could be a better more generalized solution.
Is it possible already possible or what are people thinking about  
extending the jobtracker api to query tasktracker that run a named task?
This would be a smarter solution for the described usecase, also  
people would be able to start a map runnable that runs a locking  
service and would be able to connect to that.
However I agree with Doug's point of view that centralized locking  
makes for this specific fetching use case no sense.

Any comments?

View raw message