Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 12 May 2016 14:34:12 +0000 (UTC)
From: "Steve Loughran (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12967367.1462964255000.175766.1463063652935@Atlassian.JIRA>
In-Reply-To: <JIRA.12967367.1462964255000@Atlassian.JIRA>
References: <JIRA.12967367.1462964255000@Atlassian.JIRA> <JIRA.12967367.1462964255026@arcas>
Subject: [jira] [Commented] (HADOOP-13130) s3a failures can surface as RTEs,
 not IOEs
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Thu, 12 May 2016 14:34:15 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-13130?page=3Dcom.atlassi=
an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D15=
281559#comment-15281559 ]=20

Steve Loughran commented on HADOOP-13130:
-----------------------------------------

Another example, forbidden access to a bucket.
{code}

testEncrypt256(org.apache.hadoop.fs.s3a.TestS3AEncryptionAlgorithmPropagati=
on)  Time elapsed: 0.843 sec  <<< ERROR!
com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amaz=
on S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: 409F9DDD876=
3ADE7)
=09at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpCli=
ent.java:1182)
=09at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClien=
t.java:770)
=09at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.ja=
va:489)
=09at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310=
)
=09at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3=
785)
=09at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Cl=
ient.java:1050)
=09at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Cl=
ient.java:1027)
=09at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.ja=
va:1025)
=09at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.ja=
va:82)
=09at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1432)
=09at org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:514)
=09at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:921)
=09at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:902)
=09at org.apache.hadoop.fs.contract.ContractTestUtils.writeDataset(Contract=
TestUtils.java:141)
=09at org.apache.hadoop.fs.s3a.TestS3AEncryptionAlgorithmPropagation.writeT=
henReadFile(TestS3AEncryptionAlgorithmPropagation.java:87)
=09at org.apache.hadoop.fs.s3a.TestS3AEncryptionAlgorithmPropagation.testEn=
crypt256(TestS3AEncryptionAlgorithmPropagation.java:69)
{code}

> s3a failures can surface as RTEs, not IOEs
> ------------------------------------------
>
>                 Key: HADOOP-13130
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13130
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.7.2
>            Reporter: Steve Loughran
>
> S3A failures happening in the AWS library surface as {{AmazonClientExcept=
ion}} derivatives, rather than IOEs. As the amazon exceptions are runtime e=
xceptions, any code which catches IOEs for error handling breaks.
> The fix will be to catch and wrap. The hard thing will be to wrap it with=
 meaningful exceptions rather than a generic IOE. Furthermore, if anyone ha=
s been catching AWS exceptions, they are going to be disappointed. That mea=
ns that fixing this situation could be considered "incompatible" =E2=80=94b=
ut only for code which contains assumptions about the underlying FS and the=
 exceptions they raise.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org