Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
Received-SPF: pass (athena.apache.org: domain of pipehappy@gmail.com
 designates 209.85.220.176 as permitted sender)
Message-ID: <50193D27.6060503@gmail.com>
Date: Wed, 01 Aug 2012 10:28:55 -0400
From: Yue Guan <pipehappy@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:14.0) Gecko/20120714 Thunderbird/14.0
MIME-Version: 1.0
To: user@hive.apache.org
Subject: mapper is slower than hive' mapper
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi, there

I'm writing mapreduce to replace some hive query and I find that my 
mapper is slow than hive's mapper. The Hive query is like:

select sum(column1) from table group by column2, column3;

My mapreduce program likes this:

     public static class HiveTableMapper extends Mapper<BytesWritable, 
Text, MyKey, DoubleWritable> {

         public void map(BytesWritable key, Text value, Context context) 
throws IOException, InterruptedException {
                 String[] sLine = StringUtils.split(value.toString(), 
StringUtils.ESCAPE_CHAR, HIVE_FIELD_DELIMITER_CHAR);
             context.write(new MyKey(Integer.parseInt(sLine[0]), 
sLine[1]), new DoubleWritable(Double.parseDouble(sLine[2])));
         }

     }

I assume hive is doing something similar. Is there any trick in hive to 
speed this thing up? Thank you!

Best,