hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ananth T. Sarathy" <ananth.t.sara...@gmail.com>
Subject Re: Count in shell taking hours
Date Wed, 14 Oct 2009 18:05:36 GMT
weird thing . I opened the shell in anotehr terminal and did a count with
10. My original window is still running is up to 7000. When I did the new
count it flew to 7870 and started to be slow....

here is output from client



09/10/14 12:25:26 DEBUG client.HConnectionManager$ClientZKWatcher: Got
ZooKeeper event, state: SyncConnected, type: None, path: null
09/10/14 12:25:26 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.244.9.171:58311
09/10/14 12:25:26 DEBUG client.HConnectionManager$TableServers: Found ROOT
at 10.244.9.171:58311
09/10/14 12:25:26 DEBUG client.HConnectionManager$TableServers: Cached
location address: 10.244.9.171:58311, regioninfo: REGION => {NAME =>
'.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE =>
{{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION =>
'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION =>
'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}]}}
09/10/14 12:25:27 DEBUG client.HConnectionManager$TableServers: Cached
location address: 10.244.9.171:58311, regioninfo: REGION => {NAME =>
'GS_Applications,,1255020109210', STARTKEY => '', ENDKEY => '', ENCODED =>
1732076772, TABLE => {{NAME => 'GS_Applications', FAMILIES => [{NAME =>
'Application', COMPRESSION => 'NONE', VERSIONS => '1', TTL => '2147483647',
BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME =>
'BinaryData', COMPRESSION => 'NONE', VERSIONS => '1', TTL => '2147483647',
BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME =>
'BinaryRetrieval', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}, {NAME => 'Files', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}, {NAME => 'Info', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}]}}
09/10/14 12:25:27 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.244.9.171:58311
09/10/14 12:25:27 DEBUG client.HConnectionManager$TableServers: Found ROOT
at 10.244.9.171:58311
09/10/14 12:25:27 DEBUG client.HConnectionManager$TableServers: Cached
location address: 10.244.9.171:58311, regioninfo: REGION => {NAME =>
'.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE =>
{{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION =>
'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION =>
'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}]}}
09/10/14 12:25:27 DEBUG client.HTable$ClientScanner: Creating scanner over
GS_Applications starting at key ''
09/10/14 12:25:27 DEBUG client.HTable$ClientScanner: Advancing internal
scanner to startKey at ''
09/10/14 12:25:27 DEBUG client.HConnectionManager$TableServers: Cache hit
for row <> in tableName GS_Applications: location server 10.244.9.171:58311,
location region name GS_Applications,,1255020109210

then it starts counting....


Ananth T Sarathy


On Wed, Oct 14, 2009 at 1:58 PM, stack <stack@duboce.net> wrote:

> Sounds sick. Enable DEBUG on cluster and on client.  With the latter you
> can
> see as client loads regions.  You can pass count an argument so it uses
> increments of less than 1000.  Try 10.  My guess is that its making no
> progress.
> St.Ack
>
> On Wed, Oct 14, 2009 at 10:52 AM, Ananth T. Sarathy <
> ananth.t.sarathy@gmail.com> wrote:
>
> > The count table in the shell is taking 1.5 hours to count 3000 rows so
> far.
> > It never had this problem before. Any ideas on what it's doing or how we
> > can
> > fix?
> > Ananth T Sarathy
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message