Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8593D18F81 for ; Wed, 6 Jan 2016 07:43:35 +0000 (UTC) Received: (qmail 45033 invoked by uid 500); 6 Jan 2016 07:43:35 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 44974 invoked by uid 500); 6 Jan 2016 07:43:35 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 44965 invoked by uid 99); 6 Jan 2016 07:43:35 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Jan 2016 07:43:35 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 2A901E0063; Wed, 6 Jan 2016 07:43:35 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: raviprak@apache.org To: common-commits@hadoop.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: hadoop git commit: HADOOP-12689. S3 filesystem operations stopped working correctly Date: Wed, 6 Jan 2016 07:43:35 +0000 (UTC) Repository: hadoop Updated Branches: refs/heads/branch-2 745885488 -> 633068377 HADOOP-12689. S3 filesystem operations stopped working correctly (cherry picked from commit 2d16f40dab291a29b3fc005221b12fd587615d4e) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/63306837 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/63306837 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/63306837 Branch: refs/heads/branch-2 Commit: 6330683778d684d539ae756226f62728776cc1d7 Parents: 7458854 Author: Ravi Prakash Authored: Tue Jan 5 23:26:03 2016 -0800 Committer: Ravi Prakash Committed: Tue Jan 5 23:43:17 2016 -0800 ---------------------------------------------------------------------- hadoop-common-project/hadoop-common/CHANGES.txt | 3 +++ .../java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java | 8 ++++++-- 2 files changed, 9 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/63306837/hadoop-common-project/hadoop-common/CHANGES.txt ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/CHANGES.txt b/hadoop-common-project/hadoop-common/CHANGES.txt index 45ed2c4..9ae15f2 100644 --- a/hadoop-common-project/hadoop-common/CHANGES.txt +++ b/hadoop-common-project/hadoop-common/CHANGES.txt @@ -923,6 +923,9 @@ Release 2.8.0 - UNRELEASED HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. (Dushyanth via xyao) + HADOOP-12689. S3 filesystem operations stopped working correctly + (Matt Paduano via raviprak) + Release 2.7.3 - UNRELEASED INCOMPATIBLE CHANGES http://git-wip-us.apache.org/repos/asf/hadoop/blob/63306837/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java ---------------------------------------------------------------------- diff --git a/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java b/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java index 901f89b..e5387f3 100644 --- a/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java +++ b/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java @@ -167,7 +167,7 @@ class Jets3tFileSystemStore implements FileSystemStore { return object.getDataInputStream(); } catch (S3ServiceException e) { if ("NoSuchKey".equals(e.getS3ErrorCode())) { - throw new IOException(key + " doesn't exist"); + return null; } if (e.getCause() instanceof IOException) { throw (IOException) e.getCause(); @@ -229,7 +229,11 @@ class Jets3tFileSystemStore implements FileSystemStore { OutputStream out = null; try { fileBlock = newBackupFile(); - in = get(blockToKey(block), byteRangeStart); + String blockId = blockToKey(block); + in = get(blockId, byteRangeStart); + if (in == null) { + throw new IOException("Block missing from S3 store: " + blockId); + } out = new BufferedOutputStream(new FileOutputStream(fileBlock)); byte[] buf = new byte[bufferSize]; int numRead;