hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Wong <sw...@netflix.com>
Subject RE: Best way to import complex data into Hive?
Date Thu, 06 Oct 2011 01:12:07 GMT
http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/201009.mbox/%3C4F6B25AFFFCAFE44B6259A412D5F9B101C07A390@ExchMBX104.netflix.com%3E
is a similar situation. Basically you have to use the default delimiters.


From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
Sent: Wednesday, October 05, 2011 3:34 PM
To: Hive user
Subject: Best way to import complex data into Hive?

Hi,

I have some transactional data, which has a few map elements, but one more complex - array
of maps. Something like this

CREATE EXTERNAL TABLE mytable
(
log_timestamp string,
price map <string,string>,
opt array <map<string, string>>
)
ROW FORMAT   DELIMITED
FIELDS TERMINATED BY '\001'
COLLECTION ITEMS TERMINATED BY '\004'
MAP KEYS TERMINATED BY '\002'
STORED AS TextFile;

However, the data is a bit more complex, and I also use '\003' and '\005\' for delimiters,
only I have no place to tell Hive about it.

So it seems that I can define data structures like opt array <map<string, string>>,
but I don't know how to import such data.

What would be the advice?

Thank you. Sincerely,
Mark

Mime
View raw message