hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Kerzner <mark.kerz...@shmsoft.com>
Subject Re: Hive Transform functionality using map datatype
Date Mon, 17 Oct 2011 22:36:30 GMT
Mark,

see if this post on loading complex maps in Hive may be helpful,
http://shmsoft.blogspot.com/2011/10/loading-inner-maps-in-hive.html

Mark Kerzner

On Mon, Oct 17, 2011 at 5:02 PM, Mark Grover <mgrover@oanda.com> wrote:

> Hi all,
> Thank you in advance for your help.
>
> I am using Hive Transform functionality to populate a table which has a
> column of Map<String, String> type.
>
> Here is the syntax for the reduce part:
>
> INSERT OVERWRITE TABLE
>    test_table
> PARTITION (dt = '2011-01-31')
> REDUCE
>  m.key, m.value
> USING
>  'java -classpath test.jar com.comp.bi.Reducer'
> AS
>  time, v1, v2;
>
> Table test_table is defined to by doing:
> create external table test_table (time INT, v1 STRING, v2 map<string,
> string>)
> PARTITIONED BY (dt STRING)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
> STORED AS TEXTFILE;
>
> When I run the query to populate test_table, I get an error:
> FAILED: Error in semantic analysis: Cannot insert into target table because
> column number/types are different '2011-01-31': Cannot convert column from
> string to map<string,string>.
>
> I have verified that the other columns match in number and type. If I
> convert the map<string, string> to be string type, the query doesn't error
> out.
>
> Is it possible to populate a table with a complex type (map<string,
> string>) using streaming (i.e. transform)? If so, can any of you please
> suggest how to make the query succeed?
>
> Thank you again!
>
> Mark
>
>
>
>
>
>

Mime
View raw message