hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: HDFS read stats
Date Sun, 09 Feb 2014 14:38:33 GMT
You're welcome. 

On Feb 9, 2014, at 6:27 AM, John Lilley <john.lilley@redpoint.net> wrote:

> Thanks!  I would have never found that.
> john
>  
> From: Ted Yu [mailto:yuzhihong@gmail.com] 
> Sent: Monday, January 27, 2014 4:57 PM
> To: common-user@hadoop.apache.org
> Subject: Re: HDFS read stats
>  
> FSDataInputStream has this javadoc:
> /** Utility that wraps a {@link FSInputStream} in a {@link DataInputStream}
>  
> You can utilize this method:
>   @InterfaceAudience.LimitedPrivate({"HDFS"})
>   public InputStream getWrappedStream() {
>     return in;
> And cast the return value to DFSInputStream
>  
> Cheers
>  
> 
> On Mon, Jan 27, 2014 at 11:07 AM, John Lilley <john.lilley@redpoint.net> wrote:
> Ummm… so if I’ve called FileSystem.open() with an hdfs:// path, and it returns an
FSDataInputStream, how do I get from there to the DFSInputStream that you say has the interface
I want?
> Thanks
> John
>  
>  
> From: Ted Yu [mailto:yuzhihong@gmail.com] 
> Sent: Sunday, January 26, 2014 6:16 PM
> 
> To: common-user@hadoop.apache.org
> Subject: Re: HDFS read stats
>  
> Please take a look at DFSInputStream#ReadStatistics which contains four metrics including
local bytes read.
> You can obtain ReadStatistics through getReadStatistics()
>  
> Cheers
>  
> 
> On Sun, Jan 26, 2014 at 4:00 PM, John Lilley <john.lilley@redpoint.net> wrote:
> Ted,
> Thanks for link!  I says 2.1.0 beta fix, and I can find FileSystem$Statistics class in
2.2.0 but it only seems to talk about read/write ops and bytes, not the local-vs-remote bytes.
 What am I missing?
> John
>  
>  
> From: Ted Yu [mailto:yuzhihong@gmail.com] 
> Sent: Sunday, January 26, 2014 10:26 AM
> To: common-user@hadoop.apache.org
> Subject: Re: HDFS read stats
>  
> Please take a look at https://issues.apache.org/jira/browse/HDFS-4698
>  
> Cheers
>  
> 
> On Sun, Jan 26, 2014 at 9:11 AM, John Lilley <john.lilley@redpoint.net> wrote:
> Is there a way to monitor the proportion of HDFS read data that is satisfied by local
nodes vs going across the network?
> Thanks
> John
>  
>  
>  
>  
>  

Mime
View raw message