hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-1824) want InputFormat for zip files
Date Fri, 31 Aug 2007 19:03:18 GMT
want InputFormat for zip files

                 Key: HADOOP-1824
                 URL: https://issues.apache.org/jira/browse/HADOOP-1824
             Project: Hadoop
          Issue Type: New Feature
          Components: mapred
            Reporter: Doug Cutting

HDFS is inefficient with large numbers of small files.  Thus one might pack many small files
into large, compressed, archives.  But, for efficient map-reduce operation, it is desireable
to be able to split inputs into smaller chunks, with one or more small original file per split.
 The zip format, unlike tar, permits enumeration of files in the archive without scanning
the entire archive.  Thus a zip InputFormat could efficiently permit splitting large archives
into splits that contain one or more archived files.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message