hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From barry.kap...@gmail.com
Subject Re: Question about the "N"DFSClient
Date Tue, 07 Mar 2006 19:10:46 GMT
I would like to be able to see the capacity, used, and effective bytes
of the DFS, as well as a DataNode breakdown listing name, capacity,
and remaining.

How would you feel about adding the following to DistributedFileSystem:

    public long getCapacity() throws IOException{
        return dfs.totalRawCapacity();

    public long getUsed() throws IOException{
        return dfs.totalRawUsed();

    public long getEffectiveBytes()throws IOException{
        long totalEffectiveBytes = 0;
        DFSFileInfo dfsFiles[] = dfs.listFiles(getPath(new File("/")));
        for (DFSFileInfo dfsFileInfo : dfsFiles) {
            totalEffectiveBytes += dfsFileInfo.getContentsLen();
        return totalEffectiveBytes;

    public DatanodeStats[] getDataNodeStats() throws IOException{
        DatanodeInfo[]  dnReport = dfs.datanodeReport();
        DatanodeStats[] stats = new DatnodeStats[dnReport.length];
        int i =0;

        for (DatanodeInfo datanodeInfo : dnReport) {
            stats[i].name = datanodeInfo.getName();
            stats[i].capacity = datanodeInfo.getCapacity();
            stats[i].remaining = datanodeInfo.getRemaining();
        return stats;

On 3/7/06, Doug Cutting <cutting@apache.org> wrote:
> barry.kaplan@gmail.com wrote:
> > I am going through my code and changing over from NDFS to DFS since I
> > was using Nutch primarily for the DFS. I noticed that the protection
> > of the DFSClient has changed along with things like DFSFile and
> > DataNodeInfo.Was the reason solely to make the JavaDoc simpler?
> Improving the javadoc is nice, but is a side-effect of trying to
> separate the public API from the implementation.  A smaller public API
> makes it easier to evolve the implementation without breaking folks
> (like you!).
> > I was using those objects to have a jsp rendered status check, so
> > having them available would be nice.
> What methods do you need from DFSClient that are not generically
> available on FileSystem and/or DistributedFilesystem?
> Doug

View raw message