hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-3682) when output hive table to file,users should could have a separator of their own choice
Date Thu, 11 Apr 2013 22:15:17 GMT

     [ https://issues.apache.org/jira/browse/HIVE-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sushanth Sowmyan updated HIVE-3682:
-----------------------------------

    Attachment: HIVE-3682.with.serde.patch

Hi [~caofangkun], we had a similar need, but a little further in scope than your patch, so
I've built on and modified your patch to add on a couple more features:

a) I've refactored your changes to LoadFileDesc into LocalDirectoryDesc - I thought that keeping
it in LoadFileDesc confused things a bit for future readability.
b) I've added on serde support for writing out - this allows people to use custom serdes (for
eg., HCat's JsonSerDe) when outputting.
c) I've added on support to write out to custom output formats as well, enabling the "STORED
AS" clause that exists in create table (Note that the inputformat part of the STORED-AS clause
is simply ignored as it makes no sense in this case)
                
> when output hive table to file,users should could have a separator of their own choice
> --------------------------------------------------------------------------------------
>
>                 Key: HIVE-3682
>                 URL: https://issues.apache.org/jira/browse/HIVE-3682
>             Project: Hive
>          Issue Type: New Feature
>          Components: CLI
>    Affects Versions: 0.8.1
>         Environment: Linux 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011
i686 i686 i386 GNU/Linux
> java version "1.6.0_25"
> hadoop-0.20.2-cdh3u0
> hive-0.8.1
>            Reporter: caofangkun
>            Assignee: Gang Tim Liu
>            Priority: Minor
>         Attachments: HIVE-3682-1.patch, HIVE-3682.with.serde.patch
>
>
> By default,when output hive table to file ,columns of the Hive table are separated by
^A character (that is \001).
> But indeed users should have the right to set a seperator of their own choice.
> Usage Example:
> create table for_test (key string, value string);
> load data local inpath './in1.txt' into table for_test
> select * from for_test;
> UT-01:default separator is \001 line separator is \n
> insert overwrite local directory './test-01' 
> select * from src ;
> create table array_table (a array<string>, b array<string>)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t'
> COLLECTION ITEMS TERMINATED BY ',';
> load data local inpath "../hive/examples/files/arraytest.txt" overwrite into table table2;
> CREATE TABLE map_table (foo STRING , bar MAP<STRING, STRING>)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t'
> COLLECTION ITEMS TERMINATED BY ','
> MAP KEYS TERMINATED BY ':'
> STORED AS TEXTFILE;
> UT-02:defined field separator as ':'
> insert overwrite local directory './test-02' 
> row format delimited 
> FIELDS TERMINATED BY ':' 
> select * from src ;
> UT-03: line separator DO NOT ALLOWED to define as other separator 
> insert overwrite local directory './test-03' 
> row format delimited 
> FIELDS TERMINATED BY ':' 
> select * from src ;
> UT-04: define map separators 
> insert overwrite local directory './test-04' 
> row format delimited 
> FIELDS TERMINATED BY '\t'
> COLLECTION ITEMS TERMINATED BY ','
> MAP KEYS TERMINATED BY ':'
> select * from src;

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message