Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 22 Feb 2012 03:33:07 +0000 (UTC)
From: "Jai Kumar Singh (Commented) (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: 
 <1424022458.2505.1329881587745.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <395140074.25062.1326191078581.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HBASE-5166) MultiThreaded Table Mapper
 analogous to MultiThreaded Mapper in hadoop
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213240#comment-13213240 ] 

Jai Kumar Singh commented on HBASE-5166:
----------------------------------------

@Zhihong Yu: submitted the patch for review with the suggested changes. 
For the sub prefix, I've taken this from hadoop and following the same. Reason why we are calling it SubMapRecordReader/Writer because it is intermediate RecordReader/Writer for Mapper Threads and It eventually uses RecordReader/Writer passed to MapReduce Job to do actual read/write. 

Thanks,

PS: I tried adding "Zhihong" in the reviewer list on the review page but somehow RB was failing, So I added stack as reviewer. Please do review. 

                
> MultiThreaded Table Mapper analogous to MultiThreaded Mapper in hadoop
> ----------------------------------------------------------------------
>
>                 Key: HBASE-5166
>                 URL: https://issues.apache.org/jira/browse/HBASE-5166
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Jai Kumar Singh
>            Priority: Minor
>              Labels: multithreaded, tablemapper
>         Attachments: 0001-Added-MultithreadedTableMapper-HBASE-5166.patch, 0003-Added-MultithreadedTableMapper-HBASE-5166.patch, 0005-HBASE-5166-Added-MultithreadedTableMapper.patch, 0006-HBASE-5166-Added-MultithreadedTableMapper.patch
>
>   Original Estimate: 0.5h
>  Remaining Estimate: 0.5h
>
> There is no MultiThreadedTableMapper in hbase currently just like we have a MultiThreadedMapper in Hadoop for IO Bound Jobs. 
> UseCase, webcrawler: take input (urls) from a hbase table and put the content (urls, content) back into hbase. 
> Running these kind of hbase mapreduce job with normal table mapper is quite slow as we are not utilizing CPU fully (N/W IO Bound).
> Moreover, I want to know whether It would be a good/bad idea to use HBase for these kind of usecases ?. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira