Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of brian.jeltema@digitalenvoy.net
 designates 68.64.43.136 as permitted sender)
From: Brian Jeltema <brian.jeltema@digitalenvoy.net>
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_F71BCE76-63E0-421D-B924-D11AD8EE9EC2"
Message-Id: <A24F9EFF-C451-4591-B50A-57275A58330E@digitalenvoy.net>
Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\))
Subject: Re: need help understand log output
Date: Mon, 8 Sep 2014 07:10:32 -0400
References: <3F3F4445-9642-4238-889B-E14B20D66B7B@digitalenvoy.net>
 <CALte62y9Y9PX1J27craM00iKgswMfOsY7+-MUQef7fGBG=MDbg@mail.gmail.com>
To: user@hbase.apache.org
In-Reply-To: 
 <CALte62y9Y9PX1J27craM00iKgswMfOsY7+-MUQef7fGBG=MDbg@mail.gmail.com>

--Apple-Mail=_F71BCE76-63E0-421D-B924-D11AD8EE9EC2
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252


> When number of attempts is greater than the value of
> hbase.client.start.log.errors.counter (default 9), AsyncProcess would
> produce logs cited below.
> The interval following 'retrying after ' is the backoff time.
>=20
> Which release of HBase are you using ?
>=20

HBase Version 0.98.0.2.1.1.0-385-hadoop2

The MR job is reading from  an HBase snapshot, if that=92s relevant.

> Cheers
>=20
>=20
> On Sun, Sep 7, 2014 at 8:50 AM, Brian Jeltema <
> brian.jeltema@digitalenvoy.net> wrote:
>=20
>> I have a map/reduce job that is consistently failing with timeouts. =
The
>> failing mapper log files contain a series
>> of records similar to those below. When I look at the hbase and hdfs =
logs
>> (on foo.net in this case) I don=92t see
>> anything obvious at these timestamps. The mapper task times out =
at/near
>> attempt=3D25/35. Can anyone shed light
>> on what these log entries mean?
>>=20
>> Thanks - Brian
>>=20
>>=20
>> 2014-09-07 09:36:51,421 INFO [htable-pool1-t1]
>> org.apache.hadoop.hbase.client.AsyncProcess: #3, table=3DHost, =
primary,
>> attempt=3D10/35 failed 1062 ops, last exception: null on =
foo.net,60020,1406043467187,
>> tracking started null, retrying after 10029 ms, replay 1062 ops
>> 2014-09-07 09:37:01,642 INFO [htable-pool1-t1]
>> org.apache.hadoop.hbase.client.AsyncProcess: #3, table=3DHost, =
primary,
>> attempt=3D11/35 failed 1062 ops, last exception: null on =
foo.net,60020,1406043467187,
>> tracking started null, retrying after 10023 ms, replay 1062 ops
>> 2014-09-07 09:37:12,064 INFO [htable-pool1-t1]
>> org.apache.hadoop.hbase.client.AsyncProcess: #3, table=3DHost, =
primary,
>> attempt=3D12/35 failed 1062 ops, last exception: null on =
foo.net,60020,1406043467187,
>> tracking started null, retrying after 20182 ms, replay 1062 ops
>> 2014-09-07 09:37:32,708 INFO [htable-pool1-t1]
>> org.apache.hadoop.hbase.client.AsyncProcess: #3, table=3DHost, =
primary,
>> attempt=3D13/35 failed 1062 ops, last exception: null on =
foo.net,60020,1406043467187,
>> tracking started null, retrying after 20140 ms, replay 1062 ops
>> 2014-09-07 09:37:52,940 INFO [htable-pool1-t1]
>> org.apache.hadoop.hbase.client.AsyncProcess: #3, table=3DHost, =
primary,
>> attempt=3D14/35 failed 1062 ops, last exception: null on =
foo.net,60020,1406043467187,
>> tracking started null, retrying after 20041 ms, replay 1062 ops
>> 2014-09-07 09:38:13,324 INFO [htable-pool1-t1]
>> org.apache.hadoop.hbase.client.AsyncProcess: #3, table=3DHost, =
primary,
>> attempt=3D15/35 failed 1062 ops, last exception: null on =
foo.net,60020,1406043467187,
>> tracking started null, retrying after 20041 ms, replay 1062 ops
>>=20
>>=20


--Apple-Mail=_F71BCE76-63E0-421D-B924-D11AD8EE9EC2--