hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Mackrory (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14448) Play nice with ITestS3AEncryptionSSEC
Date Tue, 06 Jun 2017 23:55:18 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sean Mackrory updated HADOOP-14448:
    Attachment: HADOOP-14448-HADOOP-13345.001.patch

Attaching patch that skips the tests that usually only throw AccessDeniedException because
of metadata operations that we short-circuit when S3Guard is enabled.

While testing this, I've seen testCreateFileThenMoveWithDifferentSSECKey and testCreateFileThenReadWithDifferentSSECKey
fail (in the sense that they fail to throw an exception) both with and without S3Guard. So
it seems unrelated to S3Guard, but I'm wondering if we're actually hitting a consistency issue,
where there's no object, or there's an old unencrypted object or something and so the test
sequence doesn't throw an AccessDeniedException? Just thinking out loud.

But this patch addresses what appears to be the only S3Guard-related part of the problem.

> Play nice with ITestS3AEncryptionSSEC
> -------------------------------------
>                 Key: HADOOP-14448
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14448
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: HADOOP-13345
>            Reporter: Sean Mackrory
>         Attachments: HADOOP-14448-HADOOP-13345.001.patch
> HADOOP-14035 hasn't yet been merged with HADOOP-13345, but it adds tests that will break
when run with S3Guard enabled. It expects that certain filesystem actions will throw exceptions
when the client-provided encryption key is not configured properly, but those actions may
sometimes bypass S3 entirely thanks to S3Guard (for example, getFileStatus may not actually
need to invoke s3GetFileStatus). If the exception is never thrown, the test fails.
> At a minimum we should tweak the tests so they definitely invoke S3 directly, or just
skip the offending tests when anything but the Null implementation is in use. This also opens
the larger question of whether or not S3Guard should be serving up metadata that is otherwise
only accessible when an encryption key is provided.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message