hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Question about the "N"DFSClient
Date Tue, 07 Mar 2006 20:01:06 GMT
This looks good to me.  +1

Can you start a bug report and add this as a patch file?

DatanodeStats looks a lot like DatanodeInfo.  Was that your intent?  I 
wouldn't mind if this was a separate class, or we could try to sanitize 
DatanodeInfo, so it does not expose, e.g., the block-related methods.

We'll also need some javadoc comments.  And we should probably re-write 
DFSShell.report() to be implemented in terms of these new methods.

Thanks,

Doug

barry.kaplan@gmail.com wrote:
> I would like to be able to see the capacity, used, and effective bytes
> of the DFS, as well as a DataNode breakdown listing name, capacity,
> and remaining.
> 
> How would you feel about adding the following to DistributedFileSystem:
> 
>     public long getCapacity() throws IOException{
>         return dfs.totalRawCapacity();
>     }
> 
>     public long getUsed() throws IOException{
>         return dfs.totalRawUsed();
>     }
> 
>     public long getEffectiveBytes()throws IOException{
>         long totalEffectiveBytes = 0;
>         DFSFileInfo dfsFiles[] = dfs.listFiles(getPath(new File("/")));
>         for (DFSFileInfo dfsFileInfo : dfsFiles) {
>             totalEffectiveBytes += dfsFileInfo.getContentsLen();
>         }
>         return totalEffectiveBytes;
>     }
> 
>     public DatanodeStats[] getDataNodeStats() throws IOException{
>         DatanodeInfo[]  dnReport = dfs.datanodeReport();
>         DatanodeStats[] stats = new DatnodeStats[dnReport.length];
>         int i =0;
> 
>         for (DatanodeInfo datanodeInfo : dnReport) {
>             stats[i].name = datanodeInfo.getName();
>             stats[i].capacity = datanodeInfo.getCapacity();
>             stats[i].remaining = datanodeInfo.getRemaining();
>             i++;
>         }
>         return stats;
>     }
> 
> On 3/7/06, Doug Cutting <cutting@apache.org> wrote:
> 
>>barry.kaplan@gmail.com wrote:
>>
>>>I am going through my code and changing over from NDFS to DFS since I
>>>was using Nutch primarily for the DFS. I noticed that the protection
>>>of the DFSClient has changed along with things like DFSFile and
>>>DataNodeInfo.Was the reason solely to make the JavaDoc simpler?
>>
>>Improving the javadoc is nice, but is a side-effect of trying to
>>separate the public API from the implementation.  A smaller public API
>>makes it easier to evolve the implementation without breaking folks
>>(like you!).
>>
>>
>>>I was using those objects to have a jsp rendered status check, so
>>>having them available would be nice.
>>
>>What methods do you need from DFSClient that are not generically
>>available on FileSystem and/or DistributedFilesystem?
>>
>>Doug
>>

Mime
View raw message