hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shantian Purkad <>
Subject Re: Passing table properties to the InputFormat
Date Mon, 22 Aug 2011 16:08:52 GMT
I have been able to get the table properties in InputFormat as below. However I am not sure
if that is correct way or if there is any better way for that.

Properties tableProperties = Utilities.getMapRedWork(job).getPathToPartitionInfo().get(getInputPaths(job)[0].toString()).getTableDesc().getProperties()

From: Shantian Purkad <>
To: "" <>
Sent: Saturday, August 20, 2011 5:01 PM
Subject: Passing table properties to the InputFormat


I have a custom Input format that reads multiple lines as one row based on number of columns
in a table.

I want to dynamically pass the table properties (like number of columns in table, their data
types etc. just like what you get in SerDe) How can I do that?

If that is not possible, and SerDe is an option, how can I use my custom record reader in

My table definition is 

create table delimited_data_serde
col1 int,
col2 string,
col3 int,
col4 string,
col5 string,
col6 string

STORED AS INPUTFORMAT 'fwrk.hadoop.input.DelimitedInputFormat'

The input format needs needs the property 'total.fields.count'='6'
If I set this using set total.fields.count=6 ; It works, however I will have to change this
property for every table that uses the custom Input format before I query that table.
How can I automatically get handle to the table properties in input format?

View raw message