hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gil Vernik (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-12092) Issues with sub-directories in Swift
Date Tue, 16 Jun 2015 08:24:01 GMT
Gil Vernik created HADOOP-12092:

             Summary: Issues with sub-directories in Swift
                 Key: HADOOP-12092
                 URL: https://issues.apache.org/jira/browse/HADOOP-12092
             Project: Hadoop Common
          Issue Type: Improvement
          Components: fs/swift
            Reporter: Gil Vernik
            Assignee: Gil Vernik

OpenStack swift doesn't have notion of directories or sub-directories. In Swift everything
is object, stored in container, that belongs to account.
Openstack Swift allows objects to have delimiters and than users can access and filter those
objects using delimiter.

Very good explanation appear here http://docs.rackspace.com/files/api/v1/cf-devguide/content/Pseudo-Hierarchical_Folders_Directories-d1e1580.html

Current driver has many code that create nested directories as zero length objects. While
it might be needed for some cases, in general this
is wrong when working with Swift and highly affects performance of the driver.

The goal of this Jira is too make "sub-directories" generation as a configurable option. There
will be option that will allow to disable sub-directories  generation and this will highly
improve the performance.

Example: client perform PUT account/container/a/b/c/d/e/f/g.txt and driver is configured not
to use sub-directories in Swift, than only one object "a/b/c/d/e/f/g.txt" will be generated
in the container.

This message was sent by Atlassian JIRA

View raw message