hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elek, Marton (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDDS-525) Support virtual-hosted style URLs
Date Thu, 20 Sep 2018 15:46:00 GMT
Elek, Marton created HDDS-525:

             Summary: Support virtual-hosted style URLs
                 Key: HDDS-525
                 URL: https://issues.apache.org/jira/browse/HDDS-525
             Project: Hadoop Distributed Data Store
          Issue Type: Bug
            Reporter: Elek, Marton

AWS supports to kind of pattern for the base url of the s3 rest api: virtual-hosted style
and path-style.

Path style: http://s3.us-east-2.amazonaws.com/bucket
Virtual-hosted style: http://bucket.s3.us-east-2.amazonaws.com

By default we support the path style method with the volume name in the url:


Here the endpoint url is http://s3.us-east-2.amazonaws.com/volume/ and the bucket is appended.

Some of the 3rd party s3 tools (goofys is an example) Supports only the virtual style method.
With goofys we can set a custom endpoint (http://localhost:9878) but all the other postfixes
after the port are removed.

It can be solved with using virtual-style url which also could include the volume name:


The easiest way is to support both of them is implementing a ContainerRequestFilter which
can parse the hostname (based on a configuration value) and extend the existing url with adding
the missing volume/bucket part. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message