hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ch huang <justlo...@gmail.com>
Subject Re: scan very slow in hbase
Date Wed, 07 Aug 2013 01:20:26 GMT
i set the cache still no help ,here is my code


        HTable table = new HTable(conf, tablename);
        Scan scan = new Scan();

        scan.setTimeRange(NumberUtils.toLong("1369108800515"),
NumberUtils.toLong("1369108810515"));

        scan.setCaching(10000);

        System.out.println("start get scan");
           ResultScanner rs = table.getScanner(scan);

           while(true){
            try {
             Result k = rs.next();
             if(k == null) break;
             System.out.println("get rowkey:" + new String(k.getRow()));
            }catch (Exception e){
             e.printStackTrace();
             break;
            }
           }



On Wed, Aug 7, 2013 at 12:29 AM, lars hofhansl <larsh@apache.org> wrote:

> In addition to that, make sure you have scanner caching enabled
> (Scan.setCaching).
> By default (in 0.94) every call to scanner.next() causes its own RPC
> request.
>
> -- Lars
>
>
>
> ________________________________
>  From: Stack <stack@duboce.net>
> To: Hbase-User <user@hbase.apache.org>
> Sent: Tuesday, August 6, 2013 7:38 AM
> Subject: Re: scan very slow in hbase
>
>
> I would suggest you search the mail archives before posting first (you will
> usually get your answer faster if you go this route).
>
> The below has been answered in the recent past.  See
> http://search-hadoop.com/m/5tk8QnhFqw
>
> Thanks,
> St.Ack
>
>
> On Tue, Aug 6, 2013 at 12:39 AM, ch huang <justlooks@gmail.com> wrote:
>
> > my workmate  tell me hbase is very slow for scan something ,i check the
> > region server find the following information,anyone can help?
> >
> >
> > 13/08/06 15:30:34 WARN ipc.HBaseServer: IPC Server listener on 60020:
> > readAndProcess threw exception java.io.IOException: Connection reset by
> > peer. Count of bytes read: 0
> > java.io.IOException: Connection reset by peer
> >         at sun.nio.ch.FileDispatcher.read0(Native Method)
> >         at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
> >         at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)
> >         at sun.nio.ch.IOUtil.read(IOUtil.java:171)
> >         at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:245)
> >         at
> >
> org.apache.hadoop.hbase.ipc.HBaseServer.channelRead(HBaseServer.java:1796)
> >         at
> >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1179)
> >         at
> >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
> >         at
> >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
> >         at
> >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
> >         at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >         at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >         at java.lang.Thread.run(Thread.java:662)
> > 13/08/06 15:30:34 ERROR regionserver.HRegionServer:
> > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
> > next(1916648340315433886, 1), rpc version=1, client version=29,
> > methodsFingerPrint=-1368823753 from 192.168.2.209:1150 after 8504 ms,
> > since
> > caller disconnected
> >         at
> >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
> >         at
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3856)
> >         at
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3776)
> >         at
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3768)
> >         at
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2513)
> >         at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> >         at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >         at java.lang.reflect.Method.invoke(Method.java:597)
> >         at
> >
> >
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
> >         at
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message