hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13514) BenchmarkThroughput.readLocalFile hangs with misconfigured BUFFER_SIZE
Date Mon, 30 Apr 2018 20:28:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459007#comment-16459007
] 

ASF GitHub Bot commented on HDFS-13514:
---------------------------------------

GitHub user eric-maynard opened a pull request:

    https://github.com/apache/hadoop/pull/374

    HDFS-13514. Avoid edge case where BUFFER_SIZE is 0

    As reported in [HDFS-13514](https://issues.apache.org/jira/browse/HDFS-13514), there is
a potential bug in the following code block:
    ```
    byte[] data = new byte[BUFFER_SIZE];
    long size = 0;
    while (size >= 0) {
      size = in.read(data);
    }
    ```
    where BUFFER_SIZE is 0
    I believe switching to a simple do/while can fix this.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/eric-maynard/hadoop trunk

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/hadoop/pull/374.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #374
    
----
commit e5c79f0ee0223c9c93ccc6f0ff104069c805ce8c
Author: Eric Maynard <emaynard@...>
Date:   2018-04-30T20:24:48Z

    HDFS-13514. Avoid edge case where BUFFER_SIZE is 0
    
    As reported in [HDFS-13514](https://issues.apache.org/jira/browse/HDFS-13514), there is
a potential bug in the following code block:
    ```
    byte[] data = new byte[BUFFER_SIZE];
        long size = 0;
        while (size >= 0) {
          size = in.read(data);
        }
    ```
    where BUFFER_SIZE is 0

----


> BenchmarkThroughput.readLocalFile hangs with misconfigured BUFFER_SIZE
> ----------------------------------------------------------------------
>
>                 Key: HDFS-13514
>                 URL: https://issues.apache.org/jira/browse/HDFS-13514
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 2.5.0
>            Reporter: John Doe
>            Priority: Minor
>
> When the BUFFER_SIZE is configured to be 0, the while loop in BenchmarkThroughput.readLocalFile
function hangs endlessly.
> This is because when the data.size (i.e., BUFFER_SIZE) is 0, the val will always be 0
by invoking val=in.read(data).
> Here is the code snippet.
> {code:java}
>   BUFFER_SIZE = conf.getInt("dfsthroughput.buffer.size", 4 * 1024);//when dfsthroughput.buffer.size
is configued to be 0
>   private void readLocalFile(Path path, String name, Configuration conf) throws IOException
{
>     System.out.print("Reading " + name);
>     resetMeasurements();
>     InputStream in = new FileInputStream(new File(path.toString()));
>     byte[] data = new byte[BUFFER_SIZE];
>     long size = 0;
>     while (size >= 0) {
>       size = in.read(data);
>     }
>     in.close();
>     printMeasurements();
>   }
> {code}
> The similar case is HDFS-13513



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message