hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10511) s3n:// incorrectly handles URLs with secret keys that contain a slash
Date Wed, 16 Apr 2014 13:24:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13971398#comment-13971398

ASF GitHub Bot commented on HADOOP-10511:

GitHub user darabos opened a pull request:


    Fix secret key handling in s3n:// (HADOOP-10511)

    I forgot to mention in the commit why I'm adding "example.com" as the default bucket.
"s3n:///" is not a valid path, and the URI constructor throws a URISyntaxException for it.
I think it is not a great idea anyway to use an invalid path in the tests. I understand the
intention of not using something that may be a real bucket, because it could result in accidental
damage to production data. But "example.com" is perfect for this, since it cannot exist.
    Let me know if you have other questions about the change. Thanks!

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/darabos/hadoop-common HADOOP-10511

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #14
commit 670926439f7f2abf098fb576f47266564ae50984
Author: Daniel Darabos <darabos.daniel@gmail.com>
Date:   2014-04-16T13:00:22Z

    Unit test demonstrating HADOOP-10511. It fails with "expected:<s3n://key:pass/word@example.com/test>
but was:<s3n://key:pass/test>".

commit d7935231d6a9094678dfe9f917ac4733aeaf864b
Author: Daniel Darabos <darabos.daniel@gmail.com>
Date:   2014-04-16T13:14:57Z

    Fix HADOOP-10511. URI.create() takes an encoded string. But URI.getAuthority() returns
a decoded string. Instead of using URI.create() we need to use the URI constructor which takes
a set of URI components.


> s3n:// incorrectly handles URLs with secret keys that contain a slash
> ---------------------------------------------------------------------
>                 Key: HADOOP-10511
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10511
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>            Reporter: Daniel Darabos
> This is similar to HADOOP-3733, but happens on s3n:// instead of s3://.
> Essentially if I have a path like "s3n://key:pass%2Fword@example.com/test", it will under
certain circumstances be replaced with "s3n://key:pass/test" which then causes "Invalid hostname
in URI" exceptions.
> I have a unit test and a fix for this. I'll make a pull request in a moment.

This message was sent by Atlassian JIRA

View raw message