hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "akshay (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-7542) Cannot import text data to Hive tables with RCFile storage (DO NOT want to use intermediate tables)
Date Tue, 29 Jul 2014 20:38:40 GMT

     [ https://issues.apache.org/jira/browse/HIVE-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

akshay updated HIVE-7542:
-------------------------

    Description: 
We plan to use RCFiles to create a data store as it can help store data in compressed format
and the columnar format enables better querying for selective columns.

Problem: When we import data from text files (comma/tab delimited) into tables with RCFile
storage format, we get an error as stated below:

"Failed with exception Wrong file format. Please check the file's format.

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask"

Workaround:
I know we can create an intermediate table.
Load data from text file to that table.
Then use insert into table rc_table select * from temp_text_file_table

But, we do not want to create intermediate tables as we have thousands of TB of data.
        Summary: Cannot import text data to Hive tables with RCFile storage (DO NOT want to
use intermediate tables)  (was: Cannot import text data to Hive tables with RCFile storage)

> Cannot import text data to Hive tables with RCFile storage (DO NOT want to use intermediate
tables)
> ---------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-7542
>                 URL: https://issues.apache.org/jira/browse/HIVE-7542
>             Project: Hive
>          Issue Type: Bug
>          Components: Compression, File Formats, HiveServer2
>            Reporter: akshay
>            Priority: Critical
>
> We plan to use RCFiles to create a data store as it can help store data in compressed
format and the columnar format enables better querying for selective columns.
> Problem: When we import data from text files (comma/tab delimited) into tables with RCFile
storage format, we get an error as stated below:
> "Failed with exception Wrong file format. Please check the file's format.
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask"
> Workaround:
> I know we can create an intermediate table.
> Load data from text file to that table.
> Then use insert into table rc_table select * from temp_text_file_table
> But, we do not want to create intermediate tables as we have thousands of TB of data.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message