hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kasi Subrahmanyam <kasisubbu...@gmail.com>
Subject Using NlineInputFormat in HIve
Date Tue, 27 Oct 2015 21:31:37 GMT
Hi,
I have a requirement where i have to send one line to one mapper in a file
but doing it using hive.
How can we implement the functionality of NLineInptFormat in Hive.
I couldnt find this and i tried the following configuration in hive

set hive.merge.mapfiles=false;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;

set mapreduce.input.fileinputformat.split.maxsize=100;

I have a file with onlt 10000 rows but i need each row to go different mapper

But if i used the above configuration in hive some of the rows are missing

based on the max split size i provide


Thanks,

Mime
View raw message