hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: Fine tunning
Date Fri, 17 Jan 2014 12:19:32 GMT
In your mapreduce, in the mapper.setup() read the lookup tables in to memory and then access
them as needed in your Mapper.map() method.
Same for reducer... 

See mapper  joins in Map/Reduce ... 

On Jan 6, 2014, at 3:23 AM, Ranjini Rathinam <ranjinibecse@gmail.com> wrote:

> Hi,
> I have a input File of 16 fields in it.
> Using Mapreduce code need to load the hbase tables.
> The first eight has to go into one table in hbase and last eight has to got
> to another hbase table.
> The data is being loaded into hbase table in 0.11 sec , but if any lookup
> is being added in the mapreduce code,
> For eg, the input file has one  attribute named currency , it will have a
> master table currency. need to match both values to print it.
> The table which has lookup takes long time to get load. For 13250 records
> it take 59 mins.
> How to make fine tune to reduce the time for its loading.
> Please help.
> Thanks in advance.
> Ranjini.R

The opinions expressed here are mine, while they may reflect a cognitive thought, that is
purely accidental. 
Use at your own risk. 
Michael Segel
michael_segel (AT) hotmail.com

View raw message