hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Namit Jain <>
Subject RE: set not work
Date Thu, 10 Jun 2010 05:20:48 GMT
use CombineHiveInputFormat

check your hive.input.format

From: Alex Kozlov []
Sent: Wednesday, June 09, 2010 9:15 PM
Subject: Re: set not work

Hi Wd,


hive.merge.size.per.task=1000000 (or some other large number)

Alex K

On Wed, Jun 9, 2010 at 6:55 PM, wd <<>> wrote:
I have lots of small files in hive, the mapred is too slow .... Is there a way to improve
the speed ?

2010/6/10 Edward Capriolo <<>>

On Wed, Jun 9, 2010 at 3:04 AM, wd <<>> wrote:
I've tried hive 0.5, the option not work too.
And find this page[]
via google.

2010/6/9 wd <<>>


I'm using hive svn rev946854. And try to set at hive cli, but seemes it
doesn't work, total map tasks still over 300+.

Is this a svn version problem?

You answered your own question, look in the link

"You cannot force but can specify mapred.reduce.tasks. "

Map tasks is based on the number of input files and folders. Even though hive uses a CombinedInput
format you still can get a number of mappers.


View raw message