hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2417) Merging of compressed rcfiles fails to write the valuebuffer part correctly
Date Wed, 31 Aug 2011 04:35:09 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094297#comment-13094297
] 

He Yongqiang commented on HIVE-2417:
------------------------------------

bq.The 'create' adds one file, and the insert adds another file.
sorry, i thought you are doing an "insert overwrite ", can u do 2 inserts? 

bq.This is needed so that the rcfiles in the target table are compressed with Bzip2. Do you
mean that we should be using Default compression codec instead? Fine with me but why is that
important?

Yes. i mean if you remove this line and keep the line "set hive.exec.compress.output = true;".
The output will be compressed using DefaultCodec. The reason is that BZip2 may not installed
for all hive users/dev.

> Merging of compressed rcfiles fails to write the valuebuffer part correctly
> ---------------------------------------------------------------------------
>
>                 Key: HIVE-2417
>                 URL: https://issues.apache.org/jira/browse/HIVE-2417
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Krishna Kumar
>            Assignee: Krishna Kumar
>         Attachments: HIVE-2417.v0.patch
>
>
> The blockmerge task does not create proper rc files when merging compressed rc files
as the valuebuffer writing is incorrect.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message