hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Grover <mgro...@oanda.com>
Subject Hive Transform functionality using map datatype
Date Mon, 17 Oct 2011 22:02:02 GMT
Hi all,
Thank you in advance for your help.

I am using Hive Transform functionality to populate a table which has a column of Map<String,
String> type.

Here is the syntax for the reduce part:

INSERT OVERWRITE TABLE
    test_table
PARTITION (dt = '2011-01-31')
REDUCE
  m.key, m.value
USING
  'java -classpath test.jar com.comp.bi.Reducer'
AS
  time, v1, v2;

Table test_table is defined to by doing:
create external table test_table (time INT, v1 STRING, v2 map<string, string>)
PARTITIONED BY (dt STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE;

When I run the query to populate test_table, I get an error:
FAILED: Error in semantic analysis: Cannot insert into target table because column number/types
are different '2011-01-31': Cannot convert column from string to map<string,string>.

I have verified that the other columns match in number and type. If I convert the map<string,
string> to be string type, the query doesn't error out.

Is it possible to populate a table with a complex type (map<string, string>) using streaming
(i.e. transform)? If so, can any of you please suggest how to make the query succeed?

Thank you again!

Mark






Mime
View raw message