hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HIVE-1343) add an interface in RCFile to support concatenation of two files without (de)compression
Date Thu, 21 Apr 2011 00:05:06 GMT

     [ https://issues.apache.org/jira/browse/HIVE-1343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

He Yongqiang resolved HIVE-1343.
--------------------------------

    Resolution: Fixed

> add an interface in RCFile to support concatenation of two files without (de)compression
> ----------------------------------------------------------------------------------------
>
>                 Key: HIVE-1343
>                 URL: https://issues.apache.org/jira/browse/HIVE-1343
>             Project: Hive
>          Issue Type: New Feature
>          Components: Serializers/Deserializers
>    Affects Versions: 0.6.0
>            Reporter: Ning Zhang
>            Assignee: He Yongqiang
>         Attachments: HIVE-1343.1.patch
>
>
> If two files are concatenated, we need to read each record in these files and write them
back to the destination file. The IO cost is mostly unavoidable due to the lack of append
functionality in HDFS. However the CPU cost could be significantly reduced by avoiding compression
and decompression of the files.
> The File Format layer should provide API that implement the block-level concatenation.


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message