Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of
 prvs=92418fa71=wlu@microstrategy.com designates 199.255.80.101 as permitted
 sender)
From: "Lu, Wei" <wlu@microstrategy.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Subject: RE: issue about search speed and rowkey
Thread-Topic: issue about search speed and rowkey
Thread-Index: AQHOk08lVqSMPru9VUy9BBgsHhDW7JmJpb0A
Date: Wed, 7 Aug 2013 12:11:35 +0000
Message-ID: 
 <489411E1EA33684CB6E6E8A81970EA82791952@PROD-EXCH-M3.corp.microstrategy.com>
References: 
 <CACeqxwQ=Vwv1T8KVadrabaNxkXfzeOLbh+XM95sHZLUUEZns1w@mail.gmail.com>
In-Reply-To: 
 <CACeqxwQ=Vwv1T8KVadrabaNxkXfzeOLbh+XM95sHZLUUEZns1w@mail.gmail.com>
Accept-Language: en-US, zh-CN
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

decrease cache size (say, 1000)  and increase batch or just set it as defau=
lt if #qualifiers in a row is not too many.


-----Original Message-----
From: ch huang [mailto:justlooks@gmail.com]=20
Sent: Wednesday, August 07, 2013 5:18 PM
To: user@hbase.apache.org
Subject: issue about search speed and rowkey

hi,all:

        i have problem in running following code,MoveData method is used to
get data from source table ,and modify each row rowkey ,and insert into
destination table,and i always get error,anyone can help?

 public static void writeRow(HTablePool htp,String tablename, String
rowkey,String cf,String col,String value) {
        try {
         HTableInterface table =3D htp.getTable(Bytes.toBytes(tablename));
            Put put =3D new Put(Bytes.toBytes(rowkey));
                put.add(Bytes.toBytes(cf),
                  Bytes.toBytes(col),
                  Long.parseLong(rowkey),
                  Bytes.toBytes(value));

                table.put(put);
            table.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

public static void MoveData(String src_t,String dest_t){

    try{
     HTable tabsrc =3D new HTable(conf, src_t);
     HTable tabdest =3D new HTable(conf, dest_t);
     tabsrc.setAutoFlush(false);
     tabdest.setAutoFlush(false);
     HTablePool tablePool =3D new HTablePool(conf, 5);
     Scan scan =3D new Scan();
     scan.setCaching(10000);
        scan.setBatch(10);
        ResultScanner rs =3D tabsrc.getScanner(scan);

           for (Result r : rs){
            ArrayList al =3D new ArrayList();
            HashMap hm =3D new HashMap();
            for (KeyValue kv : r.raw()){

             hm.put(new String(kv.getQualifier()), new
String(kv.getValue()));
             al.add(new String(kv.getQualifier()));
            }

               for (int i =3D 0; i < al.size(); i++) {

writeRow(tablePool,dest_t,hm.get("date").toString(),"info",al.get(i).toStri=
ng(),hm.get(al.get(i)).toString());
               }
         }

           rs.close();
           tabsrc.close();
           tabdest.close();

    }catch(IOException e){
     e.printStackTrace();
    }

   }


2013-08-07 16:43:31,250 WARN  [main] conf.Configuration
(Configuration.java:warnOnceIfDeprecated(824)) - hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
java.lang.RuntimeException:
org.apache.hadoop.hbase.client.ScannerTimeoutException: 123891ms passed
since the last invocation, timeout is currently set to 120000
 at
org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClie=
ntScanner.java:44)
 at com.testme.demo.HBaseTest.MoveData(HBaseTest.java:186)
 at com.testme.demo.HBaseTest.main(HBaseTest.java:314)
Caused by: org.apache.hadoop.hbase.client.ScannerTimeoutException: 123891ms
passed since the last invocation, timeout is currently set to 120000
 at
org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:283)
 at
org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClie=
ntScanner.java:41)
 ... 2 more
Caused by: org.apache.hadoop.hbase.UnknownScannerException:
org.apache.hadoop.hbase.UnknownScannerException: Name: -1792412350530007203
 at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:=
2464)
 at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp=
l.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine=
.java:320)
 at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)

 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
Source)
 at java.lang.reflect.Constructor.newInstance(Unknown Source)
 at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(Remote=
ExceptionHandler.java:96)
 at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:14=
3)
 at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:42=
)
 at
org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.ja=
va:163)
 at
org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:274)
 ... 3 more