hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars George (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1829) Make use of start/stop row in TableInputFormat
Date Sat, 10 Oct 2009 08:58:31 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764318#action_12764318
] 

Lars George commented on HBASE-1829:
------------------------------------

No, you are right Ken, in createRecordReader() it does:

{code}
Scan sc = new Scan(this.scan);
sc.setStartRow(tSplit.getStartRow());
sc.setStopRow(tSplit.getEndRow());
trr.setScan(sc);
trr.setHTable(table);
trr.init();
{code}

which sets the boundaries to the current split, while not honoring the set start/stop row.
I will have to add another row key comparison to set it to the appropriate keys. I think it
should be enough to check like this:

{code}
if (scan.getStartRow().length == 0) sc.setStartRow(tSplit.getStartRow());
if (scan.getStopRow().length == 0) sc.setStopRow(tSplit.getEndRow());
{code}

Right? I'll will check this when doing the unit tests.

> Make use of start/stop row in TableInputFormat
> ----------------------------------------------
>
>                 Key: HBASE-1829
>                 URL: https://issues.apache.org/jira/browse/HBASE-1829
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Lars George
>            Assignee: Lars George
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: HBASE-1829-v2.patch, HBASE-1829.patch
>
>
> Since we can now specify a start and stop row with the Scan that is handed to the TIF
we can reduce the splits to the regions that contain these rows. That allows to test large
MR jobs on a single region for example.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message