Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <1559829530.1240509390516.JavaMail.jira@brutus>
Date: Thu, 23 Apr 2009 10:56:30 -0700 (PDT)
From: "Billy Pearson (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-1287) Partitioner class not used in
 TableMapReduceUtil.initTableReduceJob()
In-Reply-To: <1703939054.1237983172521.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702033#action_12702033 ] 

Billy Pearson commented on HBASE-1287:
--------------------------------------

question?
the default partitioner should be hash defualt in hadoop
if we pass null as we do in line 75
and its set in line 105 as null will hadoop use default hash or will this break the partitioner?

> Partitioner class not used in TableMapReduceUtil.initTableReduceJob()
> ---------------------------------------------------------------------
>
>                 Key: HBASE-1287
>                 URL: https://issues.apache.org/jira/browse/HBASE-1287
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Lars George
>            Assignee: Billy Pearson
>             Fix For: 0.19.2, 0.20.0
>
>         Attachments: 1287-2.patch, 1287-3-patch.txt, 1287-4-patch.txt, 1287-5.patch, 1287.patch
>
>
> Upon checking the available utility methods in TableMapReduceUtil I came across this code
> {code}
>   public static void initTableReduceJob(String table,
>     Class<? extends TableReduce> reducer, JobConf job, Class partitioner)
>   throws IOException {
>     job.setOutputFormat(TableOutputFormat.class);
>     job.setReducerClass(reducer);
>     job.set(TableOutputFormat.OUTPUT_TABLE, table);
>     job.setOutputKeyClass(ImmutableBytesWritable.class);
>     job.setOutputValueClass(BatchUpdate.class);
>     if (partitioner != null) {
>       job.setPartitionerClass(HRegionPartitioner.class);
>       HTable outputTable = new HTable(new HBaseConfiguration(job), table);
>       int regions = outputTable.getRegionsInfo().size();
>       if (job.getNumReduceTasks() > regions){
>     	job.setNumReduceTasks(outputTable.getRegionsInfo().size());
>       }
>     }
>   }
> {code}
> It seems though as it should be
> {code}
>     if (partitioner != null) {
>       job.setPartitionerClass(partitioner);
> {code}
> and the provided HRegionPartitioner can be handed in to that call or a custom one can be provided.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.