Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of gzbigegg.ml@gmail.com
 designates 209.85.198.238 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=eCl/Xgd9N02danxdr/mAQny+9fQx8LST/7FH1uC5embQeW3vlX8rDLy199A9em+BZh
         LAhBBYCcTbmhjTSoENWKOFIL4Tik9E67LBbY0Ib5N6bUYUeXBPb9IAp19e8oYipKQeQr
         VGvV725FQqU8erIXqYwxhUb/cl0yTdyh8mLsc=
MIME-Version: 1.0
In-Reply-To: <7c962aed0902281441h34749c37pb42f44f16d14b2e2@mail.gmail.com>
References: <f60381460902280800g5be43d5fj6ad22930e6fadb29@mail.gmail.com>
	 <7c962aed0902281441h34749c37pb42f44f16d14b2e2@mail.gmail.com>
Date: Sun, 1 Mar 2009 16:39:30 +0800
Message-ID: <f60381460903010039m5e87dad6s120f1ce921ab30a0@mail.gmail.com>
Subject: Re: Exception when scanning table
From: Liu Yan <gzbigegg.ml@gmail.com>
To: hbase-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=000e0cd22cd8008b6404640aa44d

--000e0cd22cd8008b6404640aa44d
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

I have 2 tables, each with about 350K rows (I did "count" on them).

For the first table, the scan kept running for about 5 hours before failing
out. For the second table, it took about 5 minutes to see the same error.

How to increase the timeout, I searched for "timeout" or "scan" keyword
under the ./conf folder, didn't find anything too helpful.

Regards,
Yan

2009/3/1 stack <stack@duboce.net>

> Before you hit the exception, had it been scanning fine?
> The exception from the regionserver is a complaint that an outstanding
> scanner didn't check in within the configured client timeout.  Was
> something
> happening on the client?  Was the cell being fetched extra large?  You
> could
> up the timeout and see if that helps.
>
> St.Ack
>
>
> On Sat, Feb 28, 2009 at 8:00 AM, Liu Yan <gzbigegg.ml@gmail.com> wrote:
>
> > hi,
> >
> > I tried to scan a table with "scan 'table_name'" command in the hbase
> > shell,
> > and hit the following exception:
> >
> > {{{
> > NativeException: java.lang.RuntimeException:
> > org.apache.hadoop.hbase.client.ScannerTimeoutException
> >    from org/apache/hadoop/hbase/client/HTable.java:1704:in `hasNext'
> >    from sun.reflect.GeneratedMethodAccessor10:-1:in `invoke'
> >    from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
> >    from java/lang/reflect/Method.java:597:in `invoke'
> >    from org/jruby/javasupport/JavaMethod.java:250:in
> > `invokeWithExceptionHandling'
> >    from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
> >    from org/jruby/javasupport/JavaClass.java:416:in `execute'
> >    from
> org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in
> > `call'
> >    from org/jruby/internal/runtime/methods/DynamicMethod.java:70:in
> `call'
> >    from org/jruby/runtime/CallSite.java:295:in `call'
> >    from org/jruby/evaluator/ASTInterpreter.java:646:in `callNode'
> >    from org/jruby/evaluator/ASTInterpreter.java:324:in `evalInternal'
> >    from org/jruby/evaluator/ASTInterpreter.java:1790:in `whileNode'
> >    from org/jruby/evaluator/ASTInterpreter.java:505:in `evalInternal'
> >    from org/jruby/evaluator/ASTInterpreter.java:620:in `blockNode'
> >    from org/jruby/evaluator/ASTInterpreter.java:318:in `evalInternal'
> > ... 121 levels...
> >    from
> >
> >
> ruby.usr.local.hbase_minus_0_dot_19_dot_0.bin.hirbInvokermethod__32$RUBY$startOpt:-1:in
> > `call'
> >    from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in
> `call'
> >    from org/jruby/internal/runtime/methods/CompiledMethod.java:48:in
> `call'
> >    from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> >    from org/jruby/runtime/CallSite.java:298:in `call'
> >    from
> >
> >
> ruby/usr/local/hbase_minus_0_dot_19_dot_0/bin//usr/local/hbase/bin/../bin/hirb.rb:429:in
> > `__file__'
> >    from
> >
> >
> ruby/usr/local/hbase_minus_0_dot_19_dot_0/bin//usr/local/hbase/bin/../bin/hirb.rb:-1:in
> > `__file__'
> >    from
> >
> >
> ruby/usr/local/hbase_minus_0_dot_19_dot_0/bin//usr/local/hbase/bin/../bin/hirb.rb:-1:in
> > `load'
> >    from org/jruby/Ruby.java:512:in `runScript'
> >    from org/jruby/Ruby.java:432:in `runNormally'
> >    from org/jruby/Ruby.java:312:in `runFromMain'
> >    from org/jruby/Main.java:144:in `run'
> >    from org/jruby/Main.java:89:in `run'
> >    from org/jruby/Main.java:80:in `main'
> >    from /usr/local/hbase/bin/../bin/hirb.rb:334:in `scan'
> >    from (hbase):3:in `binding
> > }}}
> >
> > In the regionserver log file, I found this:
> >
> > {{{
> > 2009-02-28 10:50:48,685 INFO
> org.apache.hadoop.hbase.regionserver.HRegion:
> > compaction completed on region
> > 1002_profiles,113495530684109376,1235659992234 in 0sec
> > 2009-02-28 10:50:54,779 INFO
> > org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> > 310687422691357774 lease expired2009-02-28 10:50:55,869 ERROR
> > org.apache.hadoop.hbase.regionserver.HRegionServer
> > : org.apache.hadoop.hbase.UnknownScannerException: Name:
> 310687422691357774
> > 2009-02-28 10:50:55,871 INFO org.apache.hadoop.ipc.HBaseServer: IPC
> Server
> > handler 7 on 60020, call next(310687422691357774, 30) from
> > 10.254.51.127:54821: error
> > : org.apache.hadoop.hbase.UnknownScannerException: Name:
> 310687422691357774
> > org.apache.hadoop.hbase.UnknownScannerException: Name:
> > 310687422691357774        at
> > org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer
> > .java:1568)
> >        at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> >        at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >        at java.lang.reflect.Method.invoke(Method.java:597)        at
> > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
> >        at
> > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:895)
> > 2009-02-28 10:51:08,687 INFO
> org.apache.hadoop.hbase.regionserver.HRegion:
> > starting  compaction on region
> > 1002_profiles,266291625891696512,1235713612502
> > 2009-02-28 10:51:08,688 DEBUG
> org.apache.hadoop.hbase.regionserver.HStore:
> > 2131509311/fetl: no store files to compact
> > 2009-02-28 10:51:08,689 DEBUG
> org.apache.hadoop.hbase.regionserver.HStore:
> > 2131509311/pre_fetl: no store files to compact
> > 2009-02-28 10:51:08,691 DEBUG
> org.apache.hadoop.hbase.regionserver.HStore:
> > 2131509311/scored: no store files to compact
> > 2009-02-28 10:51:08,692 DEBUG
> org.apache.hadoop.hbase.regionserver.HStore:
> > 2131509311/reverse_edge: no store files to compact
> > 2009-02-28 10:51:08,692 DEBUG
> org.apache.hadoop.hbase.regionserver.HStore:
> > 2131509311/inferred: no store files to compact
> > }}}
> >
> > I didn't find any exception in the master log file.
> >
> > Can any one help?
> >
> > Regards,
> > Yan
> >
>

--000e0cd22cd8008b6404640aa44d--