Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of ananth.t.sarathy@gmail.com
 designates 209.85.221.201 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=NlBk22/L0dWxiKkYJY3s8eY62KgKlUkEwY87gBdYk201/8YqwlBYssMTMonEBr+RP6
         BG5MduDV/UHweRD+SftmloVNZkkCu5U7CuFJiv56uCV32IV4pjZX895zUYTM7kUOJXpa
         eGagMGnUYiGs+GJtwG+4PDl/erCuO0/krqi30=
MIME-Version: 1.0
In-Reply-To: <7c962aed0910141058l4c1ecfbqc4ece6bc20174483@mail.gmail.com>
References: <ad681e7f0910141052h242f58dcl20d466d83da22fd@mail.gmail.com>
	 <7c962aed0910141058l4c1ecfbqc4ece6bc20174483@mail.gmail.com>
Date: Wed, 14 Oct 2009 14:05:36 -0400
Message-ID: <ad681e7f0910141105q7aeaa572n4fe8d263244218da@mail.gmail.com>
Subject: Re: Count in shell taking hours
From: "Ananth T. Sarathy" <ananth.t.sarathy@gmail.com>
To: hbase-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001485f6c76a7ba70f0475e902eb

--001485f6c76a7ba70f0475e902eb
Content-Type: text/plain; charset=ISO-8859-1

weird thing . I opened the shell in anotehr terminal and did a count with
10. My original window is still running is up to 7000. When I did the new
count it flew to 7870 and started to be slow....

here is output from client


09/10/14 12:25:26 DEBUG client.HConnectionManager$ClientZKWatcher: Got
ZooKeeper event, state: SyncConnected, type: None, path: null
09/10/14 12:25:26 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.244.9.171:58311
09/10/14 12:25:26 DEBUG client.HConnectionManager$TableServers: Found ROOT
at 10.244.9.171:58311
09/10/14 12:25:26 DEBUG client.HConnectionManager$TableServers: Cached
location address: 10.244.9.171:58311, regioninfo: REGION => {NAME =>
'.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE =>
{{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION =>
'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION =>
'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}]}}
09/10/14 12:25:27 DEBUG client.HConnectionManager$TableServers: Cached
location address: 10.244.9.171:58311, regioninfo: REGION => {NAME =>
'GS_Applications,,1255020109210', STARTKEY => '', ENDKEY => '', ENCODED =>
1732076772, TABLE => {{NAME => 'GS_Applications', FAMILIES => [{NAME =>
'Application', COMPRESSION => 'NONE', VERSIONS => '1', TTL => '2147483647',
BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME =>
'BinaryData', COMPRESSION => 'NONE', VERSIONS => '1', TTL => '2147483647',
BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME =>
'BinaryRetrieval', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}, {NAME => 'Files', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}, {NAME => 'Info', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}]}}
09/10/14 12:25:27 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.244.9.171:58311
09/10/14 12:25:27 DEBUG client.HConnectionManager$TableServers: Found ROOT
at 10.244.9.171:58311
09/10/14 12:25:27 DEBUG client.HConnectionManager$TableServers: Cached
location address: 10.244.9.171:58311, regioninfo: REGION => {NAME =>
'.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE =>
{{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION =>
'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION =>
'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}]}}
09/10/14 12:25:27 DEBUG client.HTable$ClientScanner: Creating scanner over
GS_Applications starting at key ''
09/10/14 12:25:27 DEBUG client.HTable$ClientScanner: Advancing internal
scanner to startKey at ''
09/10/14 12:25:27 DEBUG client.HConnectionManager$TableServers: Cache hit
for row <> in tableName GS_Applications: location server 10.244.9.171:58311,
location region name GS_Applications,,1255020109210

then it starts counting....


Ananth T Sarathy


On Wed, Oct 14, 2009 at 1:58 PM, stack <stack@duboce.net> wrote:

> Sounds sick. Enable DEBUG on cluster and on client.  With the latter you
> can
> see as client loads regions.  You can pass count an argument so it uses
> increments of less than 1000.  Try 10.  My guess is that its making no
> progress.
> St.Ack
>
> On Wed, Oct 14, 2009 at 10:52 AM, Ananth T. Sarathy <
> ananth.t.sarathy@gmail.com> wrote:
>
> > The count table in the shell is taking 1.5 hours to count 3000 rows so
> far.
> > It never had this problem before. Any ideas on what it's doing or how we
> > can
> > fix?
> > Ananth T Sarathy
> >
>

--001485f6c76a7ba70f0475e902eb--