hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ishan Chhabra <ichha...@rocketfuel.com>
Subject Re: map reduce become much slower when upgrading from 0.94.11 to 0.96.2-hadoop1
Date Wed, 25 Jun 2014 23:18:56 GMT
Li Li,
Were you able to figure out the cause of this? I am seeing something
similar.


On Wed, May 7, 2014 at 10:50 PM, Li Li <fancyerii@gmail.com> wrote:

> today I upgraded hbase 0.94.11 to 0.96.2-hadoop1. I have not changed
> any client codes except replace 0.94.11 client jar to 0.96.2 's
> When with old version. when doing mapreduce task. the requests per
> seconds is about 10,000. But with new one, the value is 300. What's
> wrong with it?
> The hbase put and get is fast and Request Per Second is larger than 5,000
>
> my codes:
> List<Scan> scans = new ArrayList<Scan>();
> Scan urldbScan=new Scan();
> urldbScan.setCaching(5000);
> urldbScan.setCacheBlocks(false);
> urldbScan.setAttribute(Scan.SCAN_ATTRIBUTES_TABLE_NAME,
> HbaseTools.TB_URL_DB_BT);
> urldbScan.addFamily(HbaseTools.CF_BT);
> scans.add(urldbScan);
> Scan outLinkScan=new Scan();
> outLinkScan.setCaching(5000);
> outLinkScan.setCacheBlocks(false);
> outLinkScan.setAttribute(Scan.SCAN_ATTRIBUTES_TABLE_NAME,
> HbaseTools.TB_OUT_LINK_BT);
> outLinkScan.addFamily(HbaseTools.CF_BT);
> scans.add(outLinkScan);
> TableMapReduceUtil.initTableMapperJob(scans, Step1Mapper.class,
> BytesWritable.class,
> ScheduleData.class, job);
>



-- 
*Ishan Chhabra *| Rocket Scientist | RocketFuel Inc.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message