hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: DFS. How to read from a specific datanode
Date Thu, 07 Aug 2008 10:40:02 GMT
Kevin wrote:
> Thank you for the suggestion. I looked at DFSClient. It appears that
> chooseDataNode method decides which data node to connect to. Currently
> it chooses the first non-dead data node returned by namenode, which
> have sorted the nodes by proximity to the client. However,
> chooseDataNode is private, so overriding it seems infeasible. Neither
> are the callers of chooseDataNode public or protected.
> I need this because I do not want to trust namenode's ordering. For
> applications where network congestion is rare, we should let the
> client to decide which data node to load from.

dangerous. what happens when network congestion arrives and the apps are 
out there. Maybe it should be negotiated -namenode provides an ordered 
list and the client can choose some based on its own measurements. If 
the name node provides one only, that's the one you get to use

View raw message