Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8744D19F02 for ; Wed, 6 Apr 2016 02:53:29 +0000 (UTC) Received: (qmail 42281 invoked by uid 500); 6 Apr 2016 02:53:29 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 42166 invoked by uid 500); 6 Apr 2016 02:53:29 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 42140 invoked by uid 99); 6 Apr 2016 02:53:28 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Apr 2016 02:53:28 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 48484E0211; Wed, 6 Apr 2016 02:53:28 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: vinayakumarb@apache.org To: common-commits@hadoop.apache.org Date: Wed, 06 Apr 2016 02:53:29 -0000 Message-Id: <40a1d707c2104d6aa83cf9054ad22824@git.apache.org> In-Reply-To: <5bf08d2246c347898dc5f72d8b28549a@git.apache.org> References: <5bf08d2246c347898dc5f72d8b28549a@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [2/2] hadoop git commit: HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos() (Contributed by kanaka kumar avvaru) HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos() (Contributed by kanaka kumar avvaru) (cherry picked from commit 48ca23def1d1c28448a65238814070e79c8f4c4e) (cherry picked from commit d21bc811d82f685b0a1338bc513d9a925d305a17) (cherry picked from commit 62da8f6fa688b503ed124313bc2f6f664175d225) Conflicts: hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/783c99d2 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/783c99d2 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/783c99d2 Branch: refs/heads/branch-2.6 Commit: 783c99d292e2b0984a05bdb3fdb0100abdfc2ef9 Parents: 95b8700 Author: Vinayakumar B Authored: Wed Jun 10 11:05:58 2015 +0530 Committer: Vinayakumar B Committed: Wed Apr 6 10:52:28 2016 +0800 ---------------------------------------------------------------------- hadoop-common-project/hadoop-common/CHANGES.txt | 3 ++ .../apache/hadoop/fs/RawLocalFileSystem.java | 6 ++-- .../apache/hadoop/fs/TestLocalFileSystem.java | 34 ++++++++++++++++++++ 3 files changed, 41 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/783c99d2/hadoop-common-project/hadoop-common/CHANGES.txt ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/CHANGES.txt b/hadoop-common-project/hadoop-common/CHANGES.txt index 39f14ca..e651711 100644 --- a/hadoop-common-project/hadoop-common/CHANGES.txt +++ b/hadoop-common-project/hadoop-common/CHANGES.txt @@ -30,6 +30,9 @@ Release 2.6.5 - UNRELEASED HADOOP-12958. PhantomReference for filesystem statistics can trigger OOM (Sangjin Lee via jlowe) + HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream + with accurate .getPos() (kanaka kumar avvaru via vinayakumarb) + Release 2.6.4 - 2016-02-11 INCOMPATIBLE CHANGES http://git-wip-us.apache.org/repos/asf/hadoop/blob/783c99d2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java index 2a25da6..a0680bb 100644 --- a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java +++ b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java @@ -243,11 +243,13 @@ public class RawLocalFileSystem extends FileSystem { if (!exists(f)) { throw new FileNotFoundException("File " + f + " not found"); } - if (getFileStatus(f).isDirectory()) { + FileStatus status = getFileStatus(f); + if (status.isDirectory()) { throw new IOException("Cannot append to a diretory (=" + f + " )"); } return new FSDataOutputStream(new BufferedOutputStream( - new LocalFSFileOutputStream(f, true), bufferSize), statistics); + new LocalFSFileOutputStream(f, true), bufferSize), statistics, + status.getLen()); } @Override http://git-wip-us.apache.org/repos/asf/hadoop/blob/783c99d2/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java index ca78a8a..8946734 100644 --- a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java +++ b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java @@ -561,6 +561,40 @@ public class TestLocalFileSystem { } @Test + public void testAppendSetsPosCorrectly() throws Exception { + FileSystem fs = fileSys.getRawFileSystem(); + Path file = new Path(TEST_ROOT_DIR, "test-append"); + + fs.delete(file, true); + FSDataOutputStream out = fs.create(file); + + try { + out.write("text1".getBytes()); + } finally { + out.close(); + } + + // Verify the position + out = fs.append(file); + try { + assertEquals(5, out.getPos()); + out.write("text2".getBytes()); + } finally { + out.close(); + } + + // Verify the content + FSDataInputStream in = fs.open(file); + try { + byte[] buf = new byte[in.available()]; + in.readFully(buf); + assertEquals("text1text2", new String(buf)); + } finally { + in.close(); + } + } + + @Test public void testFileStatusPipeFile() throws Exception { RawLocalFileSystem origFs = new RawLocalFileSystem(); RawLocalFileSystem fs = spy(origFs);