Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <663977675.360.1491538539798.JavaMail.root@localhost>
References: <663977675.360.1491538539798.JavaMail.root@localhost>
From: Ted Yu <yuzhihong@gmail.com>
Date: Fri, 7 Apr 2017 01:50:34 -0700
Message-ID: <CALte62zdT9hwrPXOGDXMmVq3r-Fhkqmqc+ihoaJCvYrrUV9wUg@mail.gmail.com>
Subject: Re: Is it because the JVM memory is low?
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=f403045f031a0b078a054c8fb8dc
archived-at: Fri, 07 Apr 2017 08:50:44 -0000

--f403045f031a0b078a054c8fb8dc
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi, Tsutomu:
Do you keep memstore parameters as default ?

Can you pastebin the remaining log from master ?

See if you can give master more than 1GB of memory.

Cheers

On Thu, Apr 6, 2017 at 9:15 PM, <wakuda_tsutomu@intec.co.jp> wrote:

> Hello.
>
> I am testing HBASE now.
>
> The following problem occurred.
>
> I do not know why this problem occurred.
>
> Is it because the JVM memory is low?
> (The memory size of HBASE is the default(1GB).)
>
> Please give me on solution.
>
> (Problem)
>
> When registering data of a 700MiB file, subsequent data registration fail=
s.
>
> (Pseudo-Distributed environment)
> HBase : 1.2.4
> Hadoop : 2.7.3
> java version : 1.8.0_121
> Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
>
> (Server)
> OS:Red Hat Enterprise Linux Server release 6.7 (Santiago)
> =EF=BC=88Linux patdevsrv02 2.6.32-573.el6.x86_64=EF=BC=89
> CPU:Intel(R) Xeon(R) CPU E5640  @2.67GHz * 2
> Memory:12GB
> Swap:14GB
> Network:1Gbps
>
> (Log)
> (1) java_program.log
>
> 2017-04-07 11:33:12,056 [hconnection-0x4c05f83b-metaLookup-shared--pool2-=
t1]
> INFO  org.apache.hadoop.hbase.client.RpcRetryingCaller : Call exception,
> tries=3D10, retries=3D35, started=3D38618 ms ago, cancelled=3Dfalse, msg=
=3Drow
> 'test_table,test_key_1,99999999999999' on table 'hbase:meta' at
> region=3Dhbase:meta,,1.1588230740, hostname=3Dexample,16201,1491532079385=
,
> seqNum=3D0
> 2017-04-07 11:33:22,152 [hconnection-0x4c05f83b-metaLookup-shared--pool2-=
t1]
> INFO  org.apache.hadoop.hbase.client.RpcRetryingCaller : Call exception,
> tries=3D11, retries=3D35, started=3D48716 ms ago, cancelled=3Dfalse, msg=
=3Drow
> 'test_table,test_key_1,99999999999999' on table 'hbase:meta' at
> region=3Dhbase:meta,,1.1588230740, hostname=3Dexample,16201,1491532079385=
,
> seqNum=3D0
> 2017-04-07 11:34:00,532 [hconnection-0x4c05f83b-metaLookup-shared--pool2-=
t2]
> INFO  org.apache.hadoop.hbase.client.RpcRetryingCaller : Call exception,
> tries=3D10, retries=3D35, started=3D38270 ms ago, cancelled=3Dfalse, msg=
=3Drow
> 'test_table,test_key_1,99999999999999' on table 'hbase:meta' at
> region=3Dhbase:meta,,1.1588230740, hostname=3Dexample,16201,1491532079385=
,
> seqNum=3D0
> 2017-04-07 11:34:10,547 [hconnection-0x4c05f83b-metaLookup-shared--pool2-=
t2]
> INFO  org.apache.hadoop.hbase.client.RpcRetryingCaller : Call exception,
> tries=3D11, retries=3D35, started=3D48284 ms ago, cancelled=3Dfalse, msg=
=3Drow
> 'test_table,test_key_1,99999999999999' on table 'hbase:meta' at
> region=3Dhbase:meta,,1.1588230740, hostname=3Dexample,16201,1491532079385=
,
> seqNum=3D0
> 2017-04-07 11:34:49,088 [hconnection-0x4c05f83b-metaLookup-shared--pool2-=
t3]
> INFO  org.apache.hadoop.hbase.client.RpcRetryingCaller : Call exception,
> tries=3D10, retries=3D35, started=3D38337 ms ago, cancelled=3Dfalse, msg=
=3Drow
> 'test_table,test_key_1,99999999999999' on table 'hbase:meta' at
> region=3Dhbase:meta,,1.1588230740, hostname=3Dexample,16201,1491532079385=
,
> seqNum=3D0
> 2017-04-07 11:34:59,143 [hconnection-0x4c05f83b-metaLookup-shared--pool2-=
t3]
> INFO  org.apache.hadoop.hbase.client.RpcRetryingCaller : Call exception,
> tries=3D11, retries=3D35, started=3D48392 ms ago, cancelled=3Dfalse, msg=
=3Drow
> 'test_table,test_key_1,99999999999999' on table 'hbase:meta' at
> region=3Dhbase:meta,,1.1588230740, hostname=3Dexample,16201,1491532079385=
,
> seqNum=3D0
> 2017-04-07 11:35:37,840 [hconnection-0x4c05f83b-metaLookup-shared--pool2-=
t4]
> INFO  org.apache.hadoop.hbase.client.RpcRetryingCaller : Call exception,
> tries=3D10, retries=3D35, started=3D38256 ms ago, cancelled=3Dfalse, msg=
=3Drow
> 'test_table,test_key_1,99999999999999' on table 'hbase:meta' at
> region=3Dhbase:meta,,1.1588230740, hostname=3Dexample,16201,1491532079385=
,
> seqNum=3D0
> 2017-04-07 11:35:47,912 [hconnection-0x4c05f83b-metaLookup-shared--pool2-=
t4]
> INFO  org.apache.hadoop.hbase.client.RpcRetryingCaller : Call exception,
> tries=3D11, retries=3D35, started=3D48328 ms ago, cancelled=3Dfalse, msg=
=3Drow
> 'test_table,test_key_1,99999999999999' on table 'hbase:meta' at
> region=3Dhbase:meta,,1.1588230740, hostname=3Dexample,16201,1491532079385=
,
> seqNum=3D0
> :
> :
>
> (2) hbase-test-master-example.log
>
> 2017-04-07 11:34:38,084 INFO  [main-EventThread] coordination.SplitLogMan=
agerCoordination:
> task /hbase/splitWAL/RESCAN0000053376 entered state: DONE
> example,16000,1491532078399
> 2017-04-07 11:34:39,084 INFO  [main-EventThread] coordination.SplitLogMan=
agerCoordination:
> task /hbase/splitWAL/RESCAN0000053377 entered state: DONE
> example,16000,1491532078399
> 2017-04-07 11:34:40,084 INFO  [main-EventThread] coordination.SplitLogMan=
agerCoordination:
> task /hbase/splitWAL/RESCAN0000053378 entered state: DONE
> example,16000,1491532078399
> 2017-04-07 11:34:41,084 INFO  [main-EventThread] coordination.SplitLogMan=
agerCoordination:
> task /hbase/splitWAL/RESCAN0000053379 entered state: DONE
> example,16000,1491532078399
> 2017-04-07 11:34:42,084 INFO  [main-EventThread] coordination.SplitLogMan=
agerCoordination:
> task /hbase/splitWAL/RESCAN0000053380 entered state: DONE
> example,16000,1491532078399
> 2017-04-07 11:34:43,084 INFO  [main-EventThread] coordination.SplitLogMan=
agerCoordination:
> task /hbase/splitWAL/RESCAN0000053381 entered state: DONE
> example,16000,1491532078399
> 2017-04-07 11:34:44,082 INFO  [example,16000,1491532078399_
> splitLogManager__ChoreService_1] master.SplitLogManager: total tasks =3D =
4
> unassigned =3D 4 tasks=3D{/hbase/splitWAL/WALs%2Fexample%2C16201%
> 2C1491532079385-splitting%2Fexample%252C16201%252C1491532079385..meta.149=
1532095844.meta=3Dlast_update
> =3D -1 last_version =3D -1 cur_worker_name =3D null status =3D in_progres=
s
> incarnation =3D 0 resubmits =3D 0 batch =3D installed =3D 4 done =3D 0 er=
ror =3D 0,
> /hbase/splitWAL/WALs%2Fexample%2C16201%2C1491532079385-splitting%
> 2Fexample%252C16201%252C1491532079385..meta.1491532101210.meta=3Dlast_upd=
ate
> =3D -1 last_version =3D -1 cur_worker_name =3D null status =3D in_progres=
s
> incarnation =3D 0 resubmits =3D 0 batch =3D installed =3D 4 done =3D 0 er=
ror =3D 0,
> /hbase/splitWAL/WALs%2Fexample%2C16201%2C1491532079385-splitting%
> 2Fexample%252C16201%252C1491532079385..meta.1491532100787.meta=3Dlast_upd=
ate
> =3D -1 last_version =3D -1 cur_worker_name =3D null status =3D in_progres=
s
> incarnation =3D 0 resubmits =3D 0 batch =3D installed =3D 4 done =3D 0 er=
ror =3D 0,
>   /hbase/splitWAL/WALs%2Fexample%2C16201%2C1491532079385-splitting%
> 2Fexample%252C16201%252C1491532079385..meta.1491532101137.meta=3Dlast_upd=
ate
> =3D -1 last_version =3D -1 cur_worker_name =3D null status =3D in_progres=
s
> incarnation =3D 0 resubmits =3D 0 batch =3D installed =3D 4 done =3D 0 er=
ror =3D 0}
> :
> :
>
> Thank you.
>
> Tsutomu Wakuda
>
>

--f403045f031a0b078a054c8fb8dc--