hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: HDFS read stats
Date Mon, 27 Jan 2014 23:56:47 GMT
FSDataInputStream has this javadoc:

/** Utility that wraps a {@link FSInputStream} in a {@link DataInputStream}


You can utilize this method:

  @InterfaceAudience.LimitedPrivate({"HDFS"})

  public InputStream getWrappedStream() {

    return in;

And cast the return value to DFSInputStream


Cheers


On Mon, Jan 27, 2014 at 11:07 AM, John Lilley <john.lilley@redpoint.net>wrote:

>  Ummm… so if I’ve called FileSystem.open() with an hdfs:// path, and it
> returns an FSDataInputStream, how do I get from there to the DFSInputStream
> that you say has the interface I want?
>
> Thanks
>
> John
>
>
>
>
>
> *From:* Ted Yu [mailto:yuzhihong@gmail.com]
> *Sent:* Sunday, January 26, 2014 6:16 PM
>
> *To:* common-user@hadoop.apache.org
> *Subject:* Re: HDFS read stats
>
>
>
> Please take a look at DFSInputStream#ReadStatistics which contains four
> metrics including local bytes read.
>
> You can obtain ReadStatistics through getReadStatistics()
>
>
>
> Cheers
>
>
>
> On Sun, Jan 26, 2014 at 4:00 PM, John Lilley <john.lilley@redpoint.net>
> wrote:
>
> Ted,
>
> Thanks for link!  I says 2.1.0 beta fix, and I can find
> FileSystem$Statistics class in 2.2.0 but it only seems to talk about
> read/write ops and bytes, not the local-vs-remote bytes.  What am I missing?
>
> John
>
>
>
>
>
> *From:* Ted Yu [mailto:yuzhihong@gmail.com]
> *Sent:* Sunday, January 26, 2014 10:26 AM
> *To:* common-user@hadoop.apache.org
> *Subject:* Re: HDFS read stats
>
>
>
> Please take a look at https://issues.apache.org/jira/browse/HDFS-4698
>
>
>
> Cheers
>
>
>
> On Sun, Jan 26, 2014 at 9:11 AM, John Lilley <john.lilley@redpoint.net>
> wrote:
>
> Is there a way to monitor the proportion of HDFS read data that is
> satisfied by local nodes vs going across the network?
>
> Thanks
>
> John
>
>
>
>
>
>
>
>
>

Mime
View raw message