Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <1613526193.1235594821990.JavaMail.jira@brutus>
Date: Wed, 25 Feb 2009 12:47:01 -0800 (PST)
From: "Jonathan Gray (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Updated: (HBASE-1172) Modify TableInputFormat splitting
 algorithm to allow any number of mappers
In-Reply-To: <221285970.1233434999606.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HBASE-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Gray updated HBASE-1172:
---------------------------------

    Fix Version/s:     (was: 0.19.1)

Fix for 0.20.0

> Modify TableInputFormat splitting algorithm to allow any number of mappers
> --------------------------------------------------------------------------
>
>                 Key: HBASE-1172
>                 URL: https://issues.apache.org/jira/browse/HBASE-1172
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Jonathan Gray
>            Assignee: Jonathan Gray
>             Fix For: 0.20.0
>
>
> Currently, the number of mappers specified when using TableInputFormat is strictly followed if less than total regions on the input table.  If greater, the number of regions is used.
> This will modify the splitting algorithm to do the following:
> - Specify 0 mappers when you want # mappers = # regions
> - If you specify fewer mappers than regions, will use exactly the number you specify based on the current algorithm
> - If you specify more mappers than regions, will divide regions up by determining [start,X) [X,end).  The number of mappers will always be a multiple of number of regions.  This is so we do not have scanners spanning multiple regions.
> There is an additional issue in that the default number of mappers in JobConf is set to 1.  That means if a user does not explicitly set number of map tasks, a single mapper will be used.  I'm going to deal with that in a separate jira as the issue currently exists, there are a number of ways to implement this, and it's not required to complete this issue.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.