Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 80EF0102C8 for ; Thu, 13 Mar 2014 16:14:00 +0000 (UTC) Received: (qmail 12839 invoked by uid 500); 13 Mar 2014 16:13:51 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 12786 invoked by uid 500); 13 Mar 2014 16:13:49 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 12708 invoked by uid 99); 13 Mar 2014 16:13:47 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Mar 2014 16:13:47 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates 209.85.213.51 as permitted sender) Received: from [209.85.213.51] (HELO mail-yh0-f51.google.com) (209.85.213.51) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Mar 2014 16:13:40 +0000 Received: by mail-yh0-f51.google.com with SMTP id f10so1258209yha.10 for ; Thu, 13 Mar 2014 09:13:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=PGFbqEM3hbO82+3wIOeXKsCtDfuLHuuDslCo62WXbMU=; b=SvyWzh0oAa4YAhugtPN4FkNPVnzKxcAyJDfpPz9n5ylsq+xL9VdiDxsMniY0Fnm+Nm kxJkdHxt73djqySVu80Odpgk0/J/q7Ja/fD6c90c0ExYp54MPpnz7ZW/otqM14dviskT od0S+Mkvy1j+JuAhb6KPgKfGOa8Uea4sqLE+acY1uCcXkxmKUQnQsZ5XfzM4LT+BiscF lsmKfORHu0+IefccT+KGAiE0Sawjq7N0t2wMC6Tuq8axrShbvZxwHcdLFPkd89damU/q T1zZn/TbkZjis6XZ1wt1kOsY+pNttt3xQ6upSIzk5qvYyx9bS1SYpu/R+gSEYjiaZOfR 3RxQ== MIME-Version: 1.0 X-Received: by 10.236.30.2 with SMTP id j2mr3311723yha.73.1394727198917; Thu, 13 Mar 2014 09:13:18 -0700 (PDT) Received: by 10.170.79.86 with HTTP; Thu, 13 Mar 2014 09:13:18 -0700 (PDT) In-Reply-To: <798557B0204DED49937B7B1D7F6893C7015DECA6@DM2PRD9702MB007.062d.mgd.msft.net> References: <798557B0204DED49937B7B1D7F6893C7015DE7A3@DM2PRD9702MB007.062d.mgd.msft.net> <798557B0204DED49937B7B1D7F6893C7015DE7E1@DM2PRD9702MB007.062d.mgd.msft.net> <798557B0204DED49937B7B1D7F6893C7015DEA7D@DM2PRD9702MB007.062d.mgd.msft.net> <798557B0204DED49937B7B1D7F6893C7015DEB32@DM2PRD9702MB007.062d.mgd.msft.net> <798557B0204DED49937B7B1D7F6893C7015DEB7F@DM2PRD9702MB007.062d.mgd.msft.net> <798557B0204DED49937B7B1D7F6893C7015DECA6@DM2PRD9702MB007.062d.mgd.msft.net> Date: Thu, 13 Mar 2014 09:13:18 -0700 Message-ID: Subject: Re: Cannot find row in .META. for table From: Ted Yu To: Umesh Chaudhary Cc: "user@hbase.apache.org" Content-Type: multipart/alternative; boundary=089e011829cc40134d04f47f3b2b X-Virus-Checked: Checked by ClamAV on apache.org --089e011829cc40134d04f47f3b2b Content-Type: text/plain; charset=ISO-8859-1 I don't see thrift-specific parameters in the list below. Please take a look at: http://hbase.apache.org/book.html#hbase_default_configurations starting with hbase.thrift.minWorkerThreads If you have the luxury of trying out Java client, you would be able to tell whether the slowness was specific to thrift. In your first email, you mentioned 0.94.1 - please consider upgrading to 0.94.17, there have been ton of improvements since 0.94.1 Now that you're able to connect, I suggest wrapping up this thread. Feel free to ask thrift performance questions under a different subject. Cheers On Thu, Mar 13, 2014 at 12:48 AM, Umesh Chaudhary wrote: > Hi Ted, > > Thanks for the help. Now I am able to scan Hbase tables with Thrift API > but I am getting 1015 rows in 4 seconds. > > I am using hadoop 1.2.1 with hbase 0.94. I have 4 region servers(data > nodes) and 1 Hmaster(namenode) all have 4 GB RAM. I have configured on > Hbase side as below: > > > > > > hbase.cluster.distributed > > true > > > > > > > > hbase.zookeeper.distributed > > true > > > > > > > > zookeeper.session.timeout > > 1200000 > > > > > > > > hbase.zookeeper.property.tickTime > > 6000 > > > > > > > > hbase.client.scanner.caching > > 500 > > Number of rows that will be fetched when calling next > > on a scanner if it is not served from (local, client) memory. Higher > > caching values will enable faster scanners but will eat up more memory > > and some calls of next may take longer and longer times when the cache > is empty. > > Do not set this value such that the time between invocations is greater > > than the scanner timeout; i.e. hbase.regionserver.lease.period > > > > > > > > > > hbase.storescanner.parallel.seek.enable > > true > > > > > > > > hbase.hregion.majorcompaction > > 0 > > > > > > > > hbase.regionserver.handler.count > > 100 > > > > > > > > hfile.min.blocksize.size > > 65536 > > > > > > > > ipc.server.tcpnodelay > > true > > > > > > > > hfile.block.cache.size > > 0.4 > > > > > > > > hbase.regionserver.global.memstore.upperLimit > > 0.4 > > Maximum size of all memstores in a region server before new > > updates are blocked and flushes are forced. Defaults to 40% of heap. > > Updates are blocked and flushes are forced until size of all memstores > > in a region server hits hbase.regionserver.global.memstore.lowerLimit. > > > > > > > > > > hbase.hstore.blockingStoreFiles > > 30 > > > > > > > > hbase.ipc.client.tcpnodelay > > true > > > > > > hbase.block.cache.size > > 0 > > > > > > By Any means can I get better performance from Thrift? Please Advise. > > > > *From:* Ted Yu [mailto:yuzhihong@gmail.com] > *Sent:* Wednesday, March 12, 2014 10:18 PM > *To:* Umesh Chaudhary > *Cc:* user@hbase.apache.org > > *Subject:* Re: Cannot find row in .META. for table > > > > Looking at src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java : > > > > filterHashMap.put("SingleColumnValueFilter", > ParseConstants.FILTER_PACKAGE + "." + > > "SingleColumnValueFilter"); > > > > SingleColumnValueFilter should be supported. > > > > Can you show the complete stack trace ? > > > > On Tue, Mar 11, 2014 at 11:04 PM, Umesh Chaudhary > wrote: > > Hi TED, > > Now, I can successfully scan tables from Hbase but when I am using Filter > like : > > > > scanFilter.FilterString = GetBytesFromString("SingleColumnValueFilter(' ', > 'COND_P', = , 'binary:0.0')"); { I have given as Column family } > > > > I an getting this error: > > " java.lang.IllegalArgumentException: Filter Name SingleColumnValueFilter > not supported", but I need SingleColumnValueFilter due to my requirement. > > > > Please help that how can I use filters with scannerOpen? > > > > > > > > *From:* Ted Yu [mailto:yuzhihong@gmail.com] > *Sent:* Wednesday, March 12, 2014 8:37 AM > *To:* Umesh Chaudhary; user@hbase.apache.org > > > *Subject:* Re: Cannot find row in .META. for table > > > > Adding back user@ > > > > Can you look at the example in this post and compose startRow accordingly ? > > > > > http://stackoverflow.com/questions/18040012/what-is-the-equivalent-of-javas-bytebuffer-wrap-in-c > > > > On Tue, Mar 11, 2014 at 7:19 PM, Umesh Chaudhary > wrote: > > Hi Ted, > > By giving null to attributes, scanner is working now, thanks for the idea. > > But, when I give Guid.Empty.ToByteArray() as ByteBuffer startRow > parametes's value, I am getting no rows in scannerGet_result. > > Please let me know what value should I pass for start row parameter. > > . > > > > > > > > *From:* Ted Yu [mailto:yuzhihong@gmail.com] > *Sent:* Tuesday, March 11, 2014 10:15 PM > *To:* Umesh Chaudhary > > > *Subject:* Re: Cannot find row in .META. for table > > > > Have you seen this ? > > > > > http://stackoverflow.com/questions/10078348/byte-collection-based-similar-with-bytebuffer-from-java > > > > Looking > at src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java : > > > > private static void addAttributes(OperationWithAttributes op, > > Map attributes) { > > if (attributes == null || attributes.size() == 0) { > > return; > > > > You can pass C# equivalent of null for attributes. > > > > Cheers > > On Tue, Mar 11, 2014 at 8:19 AM, Umesh Chaudhary > wrote: > > Thanks for the reply Ted. I am using Hbase-sharp dll which is portd with > kind-of old scannerOpen() method which has no Map > attributes parameter. > I have also generated c# code from new Thrift server in which I am getting > 4 arguments as you have listed. > Now , my concern is in what way I should give {ByteBuffer startRow} and > {Map attributes } parameter because I want to get > all rows from the specified table. > > > > -----Original Message----- > From: Ted Yu [mailto:yuzhihong@gmail.com] > Sent: Tuesday, March 11, 2014 8:21 PM > To: user@hbase.apache.org > Subject: Re: Cannot find row in .META. for table > > In src/main//java/org/apache/hadoop/hbase/thrift/generated/Hbase.java , I > found the following scannerOpen() methods: > > public int scannerOpen(ByteBuffer tableName, ByteBuffer startRow, > List columns, Map attributes) throws > IOError, org.apache.thrift.TException; > public void scannerOpen(ByteBuffer tableName, ByteBuffer startRow, > List columns, Map attributes, > org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > public int scannerOpen(ByteBuffer tableName, ByteBuffer startRow, > List columns, Map attributes) throws > IOError, org.apache.thrift.TException > public void scannerOpen(ByteBuffer tableName, ByteBuffer startRow, > List columns, Map attributes, > org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException { > > None of the above takes 3 parameters. > > > On Tue, Mar 11, 2014 at 6:05 AM, Umesh Chaudhary >wrote: > > > I am getting below message while running hbck with/without parameters: > > > > Number of regions: 7 > > Deployed on: jci0.jci.com,60020,1394472660266 > > jci1.jci.com,60020,1394472671945 > > jci2.jci.com,60020,1394472679477 jci3.jci.com,60020,1394472703951 > > 0 inconsistencies detected. > > > > If there are 0 inconsistencies then why I am facing this issue? > > Please check my code: > > > > var rows = _hbase.getRow(table_name, > > BitConverter.GetBytes("Asset"));---> > > where "Asset" is my column family. > > > > OR > > > > var scanner = > > _hbase.scannerOpen(table_name,BitConverter.GetBytes(1),columnsListinBy > > teArray); > > > > Because I am newbie to Thrift API for C#, please suggest how can I > > provide arguments for the same. > > > > > > -----Original Message----- > > From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org] > > Sent: Tuesday, March 11, 2014 5:13 PM > > To: user > > Subject: Re: Cannot find row in .META. for table > > > > Before using -repair or any other parameter, I will recommend you to > > run it without any parameter to have a sense of what hbck will find. > > > > JM > > > > > > 2014-03-11 7:36 GMT-04:00 divye sheth : > > > > > You can use the hbck utility to repair these kinds of problems. > > > > > > $ hbase hbck -repair > > > OR > > > $ hbase hbck -fixMeta > > > > > > Thanks > > > Divye Sheth > > > > > > > > > On Tue, Mar 11, 2014 at 4:55 PM, Umesh Chaudhary > > > > > >wrote: > > > > > > > > > > > Hi, > > > > I am using Hbase 0.94.1 with Hadoop 1.2.1 and using Thrift API to > > > > access tables stored in Hbase from my C# application. I am able to > > > > connect to Server but while going to perform any operation from > > > > client it gives following error in CLI-log: > > > > > > > > 14/03/11 12:18:53 WARN > > > > client.HConnectionManager$HConnectionImplementation: Encountered > > > > problems when prefetch META table: > > > > org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in > > .META. > > > > for table: tblAssetsView, > > > > > > > row=t\x00\x00\x00b\x00\x00\x00l\x00\x00\x00A\x00\x00\x00s\x00\x00\x0 > > > 0s > > > \x00\x00\x00e\x00\x00\x00t\x00\x00\x00s\x00\x00\x00V\x00\x00\x00i\x0 > > > 0\ > > > x00\x00e\x00\x00\x00w\x00\x00\x00,,99999999999999 > > > > at > > > > > > org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:1 > > 51) > > > > at > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImpleme > > > nt > > > ation.prefetchRegionCache(HConnectionManager.java:1059) > > > > at > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImpleme > > > nt > > > ation.locateRegionInMeta(HConnectionManager.java:1121) > > > > at > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImpleme > > > nt > > > ation.locateRegion(HConnectionManager.java:1001) > > > > at > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImpleme > > > nt > > > ation.locateRegion(HConnectionManager.java:958) > > > > at > > org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) > > > > at org.apache.hadoop.hbase.client.HTable.(HTable.java:155) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getTa > > > bl > > > e(ThriftServerRunner.java:458) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getTa > > > bl > > > e(ThriftServerRunner.java:464) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getRo > > > wW > > > ithColumnsTs(ThriftServerRunner.java:766) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getRo > > > w( > > > ThriftServerRunner.java:739) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > at > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl > > > .j > > > ava:57) > > > > at > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcce > > > ss > > > orImpl.java:43) > > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(Hbase > > > Ha > > > ndlerMetricsProxy.java:65) > > > > at com.sun.proxy.$Proxy6.getRow(Unknown Source) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$getRow.getR > > > es > > > ult(Hbase.java:3906) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$getRow.getR > > > es > > > ult(Hbase.java:3894) > > > > at > > org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) > > > > at > org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnne > > > ct > > > ion.run(TBoundedThreadPoolServer.java:287) > > > > at > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor > > > .j > > > ava:1145) > > > > at > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. > > > java:615) > > > > at java.lang.Thread.run(Thread.java:744) > > > > 14/03/11 12:18:53 WARN thrift.ThriftServerRunner$HBaseHandler: > > > > tblAssetsView > > > > org.apache.hadoop.hbase.TableNotFoundException: tblAssetsView > > > > at > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImpleme > > > nt > > > ation.locateRegionInMeta(HConnectionManager.java:1139) > > > > at > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImpleme > > > nt > > > ation.locateRegion(HConnectionManager.java:1001) > > > > at > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImpleme > > > nt > > > ation.locateRegion(HConnectionManager.java:958) > > > > at > > org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) > > > > at org.apache.hadoop.hbase.client.HTable.(HTable.java:155) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getTa > > > bl > > > e(ThriftServerRunner.java:458) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getTa > > > bl > > > e(ThriftServerRunner.java:464) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getRo > > > wW > > > ithColumnsTs(ThriftServerRunner.java:766) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getRo > > > w( > > > ThriftServerRunner.java:739) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > at > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl > > > .j > > > ava:57) > > > > at > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcce > > > ss > > > orImpl.java:43) > > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(Hbase > > > Ha > > > ndlerMetricsProxy.java:65) > > > > at com.sun.proxy.$Proxy6.getRow(Unknown Source) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$getRow.getR > > > es > > > ult(Hbase.java:3906) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$getRow.getR > > > es > > > ult(Hbase.java:3894) > > > > at > > org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) > > > > at > org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) > > > > at > > > > > > > org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnne > > > ct > > > ion.run(TBoundedThreadPoolServer.java:287) > > > > at > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor > > > .j > > > > ava:1145) > > > > at > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. > > > java:615) > > > > at java.lang.Thread.run(Thread.java:744) > > > > > > > > > > > > > > > > But I can access my table from Hbase Shell with all shell > > > > operations. I > > > am > > > > totally stuck here, please devise some methods to overcome this > issue. > > > > > > > > > > > > Umesh Chaudhary > > > > > > > > > > > > > > > > --089e011829cc40134d04f47f3b2b--