Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of eychih@gmail.com designates
 209.85.160.41 as permitted sender)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1084)
Subject: Re: improve performance of a MapReduce job with HBase input
From: Ey-Chih chow <eychih@gmail.com>
In-Reply-To: 
 <CAN_xCOdeCC4C0vdxe5A_vLGzv4x28OCpmZi5d+=BpRU+byrdPw@mail.gmail.com>
Date: Fri, 25 May 2012 13:49:19 -0700
Content-Transfer-Encoding: 7bit
Message-Id: <C962CB5C-A07B-4B00-872D-D4DB21465F06@gmail.com>
References: <0ADBD8BB-4DFC-43B9-A05A-1593CEC32E0A@gmail.com>
 <CAN_xCOdeCC4C0vdxe5A_vLGzv4x28OCpmZi5d+=BpRU+byrdPw@mail.gmail.com>
To: user@hbase.apache.org

Thanks.  This help.

Ey-Chih Chow

On May 25, 2012, at 11:23 AM, Dave Revell wrote:

> Here's what I do:
> 
> Scan scan = new Scan(...)
> scan.setCaching(5000);
> scan.setWhatever(...);
> 
> TableMapReduceUtil.initTableMapperJob(tablename, scan, mapClass,
>                    mapOutKeyClass, mapOutValueClass, job);
> 
> Does that help?
> 
> -Dave
> 
> On Fri, May 25, 2012 at 11:03 AM, Ey-Chih chow <eychih@gmail.com> wrote:
> 
>> Hi,
>> 
>> We have a MapReduce job of which input data is from HBase.  We would like
>> to improve performance of the job.  According to the HBase book, we can do
>> that by setting scan caching to a number higher than default.  We use
>> TableInputFormat to read data from the job.  I look at the implementation
>> of the class.  The class does not set caching when a scan object is
>> created.  Is there anybody know how to externally set caching for the scan
>> created in TableInputFormat?  Thanks.
>> 
>> Ey-Chih Chow