Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 25 Apr 2012 13:48:18 +0000 (UTC)
From: "Ravi Prakash (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <134763490.295.1335361698527.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <163421014.454.1335304446993.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-3318) Hftp hangs on transfers >2GB
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261645#comment-13261645 ] 

Ravi Prakash commented on HDFS-3318:
------------------------------------

Hi Daryn,

Can streamLength > 2^31 ? Would in then be bounded by that higher number and still cause issues? 

Why are you Overriding 
{noformat}
  @Override
  public int read(byte[] b) throws IOException {
    return read(b, 0, b.length);
  }
{noformat}

Wouldn't the original implementation be picked up from InputStream which has exactly the same code? I tested with this short program and it prints exactly what Michael Jackson used to say he is.

{noformat}
class A {
  public void printA() {
    System.out.println("A");
    printC();
  }
  public void printC() {
    System.out.println("C");
  }
}

class B extends A {
  @Override
  public void printC() {
    System.out.println("D");
  }
  
  public void printB() {
    System.out.println("B");
    printA();
  }
}

public class TestJAVA {
  public static void main(String arg[]) {
    B b = new B();
    b.printB();
  }
}
{noformat}


> Hftp hangs on transfers >2GB
> ----------------------------
>
>                 Key: HDFS-3318
>                 URL: https://issues.apache.org/jira/browse/HDFS-3318
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 0.24.0, 0.23.3, 2.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HDFS-3318.patch
>
>
> Hftp transfers >2GB hang after the transfer is complete.  The problem appears to be caused by java internally using an int for the content length.  When it overflows 2GB, it won't check the bounds of the reads on the input stream.  The client continues reading after all data is received, and the client blocks until the server times out the connection -- _many_ minutes later.  In conjunction with hftp timeouts, all transfers >2G fail with a read timeout.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira