hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Li <fancye...@gmail.com>
Subject Re: last mapper of mapreduce on hbase very slow
Date Fri, 16 May 2014 05:53:31 GMT
how to do this? I can add some log in my map reduce program to record
pages with many links.
btw. does the order of add Scan affect it?
1.
List<Scan> scans = new ArrayList<Scan>();
scans.add(urldbScan);
scans.add(outLinkScan);
2.
List<Scan> scans = new ArrayList<Scan>();
scans.add(outLinkScan);
scans.add(urldbScan);

will code snipplet 2 be better?

On Thu, May 15, 2014 at 1:29 AM, Stack <stack@duboce.net> wrote:
> On Tue, May 13, 2014 at 6:45 PM, Li Li <fancyerii@gmail.com> wrote:
> ...
>
>> I found that at the beginning, the map-reduce will make about 3,000
>> hbase requests per second. But when there is only the last mapper
>> running, the hbase requests is less than 200 per second.
>>
>>
> Can you figure out what part of the key space is being accessed when the
> rate is low?  E.g. are these pages w/ lots of links?
> St.Ack

Mime
View raw message