Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
From: Doug Meil <doug.meil@explorysmedical.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Date: Tue, 21 Jun 2011 22:31:55 -0400
Subject: RE: TableOutputFormat not efficient than direct HBase API calls?
Thread-Topic: TableOutputFormat not efficient than direct HBase API calls?
Thread-Index: Acwwgv/VRteHKZWgQvSmTvXiwR50UwAATp4Q
Message-ID: <67680900F79B1D4F99C844EE386FC5952823CD86C9@EX2K7VS03.4emm.local>
References: <BANLkTim6WhsGXNAwPEF74nFguhxHeUaU+w@mail.gmail.com>
In-Reply-To: <BANLkTim6WhsGXNAwPEF74nFguhxHeUaU+w@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0


TableOutputFormat also does this...

    table.setAutoFlush(false);

Check out the HBase book for how the writebuffer works with the HBase clien=
t.

http://hbase.apache.org/book.html#client


-----Original Message-----
From: edward choi [mailto:mp2893@gmail.com]=20
Sent: Tuesday, June 21, 2011 10:23 PM
To: common-user@hadoop.apache.org; user@hbase.apache.org
Subject: TableOutputFormat not efficient than direct HBase API calls?

Hi,

I am writing an Hadoop application that uses HBase as both source and sink.

There is no reducer job in my application.

I am using TableOutputFormat as the OutputFormatClass.

I read it on the Internet that it is experimentally faster to directly inst=
antiate HTable and use HTable.batch() in the Map than to use TableOutputFor=
mat as the Map's OutputClass

So I looked into the source code,
org.apache.hadoop.hbase.mapreduce.TableOutputFormat.
It looked like TableRecordWriter does not support batch updates, since
TableRecordWriter.write() called HTable.put(new Put()).

Am I right on this matter? Or does TableOutputFormat automatically do batch=
 updates somehow?
Or is there a specific way to do batch updates with TableOutputFormat?

Any explanation is greatly appreciated.

Ed