Return-Path: Delivered-To: apmail-hbase-user-archive@www.apache.org Received: (qmail 26302 invoked from network); 3 Sep 2010 07:27:12 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 3 Sep 2010 07:27:12 -0000 Received: (qmail 50465 invoked by uid 500); 3 Sep 2010 07:27:11 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 50256 invoked by uid 500); 3 Sep 2010 07:27:08 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 50248 invoked by uid 99); 3 Sep 2010 07:27:07 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Sep 2010 07:27:07 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jinsong_hu@hotmail.com designates 65.55.111.95 as permitted sender) Received: from [65.55.111.95] (HELO blu0-omc2-s20.blu0.hotmail.com) (65.55.111.95) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Sep 2010 07:27:02 +0000 Received: from BLU147-DS11 ([65.55.111.72]) by blu0-omc2-s20.blu0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Fri, 3 Sep 2010 00:26:41 -0700 X-Originating-IP: [99.90.69.102] X-Originating-Email: [jinsong_hu@hotmail.com] Message-ID: From: "Jinsong Hu" To: References: <2D6136772A13B84E95DF6DA79E85A9F0011D292DD208@NSPEXMBX-A.the-lab.llnl.gov> In-Reply-To: Subject: Re: thrift for hbase in CDH3 broken ? Date: Fri, 3 Sep 2010 00:26:44 -0700 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal Importance: Normal X-Mailer: Microsoft Windows Live Mail 14.0.8089.726 X-MimeOLE: Produced By Microsoft MimeOLE V14.0.8089.726 X-OriginalArrivalTime: 03 Sep 2010 07:26:41.0987 (UTC) FILETIME=[5ADD6930:01CB4B39] are you using CDH3 distribution ? Jinsong -------------------------------------------------- From: "Alexey Kovyrin" Sent: Friday, September 03, 2010 12:04 AM To: Subject: Re: thrift for hbase in CDH3 broken ? > http://github.com/kovyrin/hbase-thrift-client-examples - just wrote > this example and tested it in our cluster, works as expected. > For this to work you'd need to install rubygems and thrift gem (gem > install thrift). > > On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu > wrote: >> Can you send me some ruby test code and so I can try against the latest >> CDH3 >> ? >> >> Jimmy. >> >> -------------------------------------------------- >> From: "Alexey Kovyrin" >> Sent: Thursday, September 02, 2010 8:15 PM >> To: >> Subject: Re: thrift for hbase in CDH3 broken ? >> >>> We use it in Scribd.com. All clients are ruby web apps. >>> >>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon wrote: >>>> >>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu >>>> wrote: >>>> >>>>> Yes, I confirmed that it is indeed thrift server. >>>>> >>>>> and the fact that the API >>>>> >>>>> List tableNamesList=client.getTableNames(); >>>>>>> >>>>>>> for (byte [] name : tableNamesList) >>>>>>> { >>>>>>> System.out.println(new String(name)); >>>>>>> } >>>>>>> >>>>>> >>>>> successfully printed all table names shows that it is indeed thrift >>>>> server. >>>>> >>>>> if it is hue, it won't print the table names. >>>>> >>>>> Ah, sorry, I missed that in your original message. Not sure what's up, >>>>> then >>>> >>>> - we don't have any changes in CDH that would affect this. Anyone here >>>> used >>>> thrift on 0.89.20100621? >>>> >>>> -Todd >>>> >>>> >>>> >>>>> Jimmy. >>>>> >>>>> -------------------------------------------------- >>>>> From: "Todd Lipcon" >>>>> Sent: Thursday, September 02, 2010 5:18 PM >>>>> >>>>> To: >>>>> Subject: Re: thrift for hbase in CDH3 broken ? >>>>> >>>>> >>>>> Hi Jinsong, >>>>>> >>>>>> Are you sure that the port you're connecting to is indeed the thrift >>>>>> server? >>>>>> >>>>>> Unfortunately both the HBase thrift server and the Hue namenode >>>>>> plugin >>>>>> listen on port 9090, so you might be having an issue where your HBase >>>>>> client >>>>>> is trying to connect to the Namenode server instead of HBase. >>>>>> >>>>>> You can verify the ports using a command like "/sbin/fuser -n tcp >>>>>> 9090" >>>>>> to >>>>>> see which pid has it open, then cross reference against sudo jps. >>>>>> >>>>>> Thanks >>>>>> -Todd >>>>>> >>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu >>>>>> wrote: >>>>>> >>>>>> Hi, There, >>>>>>> >>>>>>> I am trying to test and see if thrift for hbase works. I followed >>>>>>> the >>>>>>> example from >>>>>>> >>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe >>>>>>> http://incubator.apache.org/thrift/ >>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi >>>>>>> >>>>>>> and wrote test code: I found that client.getTableNames(); >>>>>>> returns all table names successfully, but the scanner never returned >>>>>>> any record. and it even throws exception: >>>>>>> >>>>>>> org.apache.thrift.TApplicationException: get failed: unknown result >>>>>>> at >>>>>>> >>>>>>> >>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785) >>>>>>> at >>>>>>> >>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750) >>>>>>> at >>>>>>> >>>>>>> >>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73) >>>>>>> at >>>>>>> >>>>>>> >>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128) >>>>>>> >>>>>>> I checked the syntax and the table and confirmed that the table does >>>>>>> have >>>>>>> record and it should >>>>>>> return result, but it doesn't. Can anybody tell me what is wrong ? >>>>>>> >>>>>>> At this time, I am suspecting the thrift server shipped with CDH3 >>>>>>> may >>>>>>> have >>>>>>> problem. if there anyway >>>>>>> I can get this test code working ? >>>>>>> >>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that the >>>>>>> code >>>>>>> generated from hbase-thrift project >>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift. >>>>>>> >>>>>>> >>>>>>> Jimmy. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> public void testThriftAPI() >>>>>>> { >>>>>>> // Make socket >>>>>>> TSocket transport = new TSocket("10.20.12.13", 9090); >>>>>>> >>>>>>> >>>>>>> >>>>>>> // Wrap in a protocol >>>>>>> TBinaryProtocol protocol = new TBinaryProtocol(transport); >>>>>>> >>>>>>> Hbase.Client client = new Hbase.Client(protocol); >>>>>>> >>>>>>> try { >>>>>>> transport.open(); >>>>>>> List tableNamesList=client.getTableNames(); >>>>>>> for (byte [] name : tableNamesList) >>>>>>> { >>>>>>> System.out.println(new String(name)); >>>>>>> } >>>>>>> >>>>>>> String tableName="HEARTBEAT_CLUSTER"; >>>>>>> String startRow=""; >>>>>>> List columns= new ArrayList(); >>>>>>> columns.add("fields:time_format".getBytes()); >>>>>>> columns.add("fields:customer_id".getBytes()); >>>>>>> >>>>>>> int aScannerId=0; >>>>>>> try { >>>>>>> >>>>>>> TCell cell=client.get(tableName.getBytes(), >>>>>>> "2010-08\tproduction-2".getBytes(), >>>>>>> "fields:customer_id".getBytes()); >>>>>>> if (cell != null) >>>>>>> { >>>>>>> String value= new String( cell.value); >>>>>>> System.out.println(value); >>>>>>> } >>>>>>> aScannerId=client.scannerOpen(tableName.getBytes(), >>>>>>> startRow.getBytes(), columns); >>>>>>> TRowResult result=null; >>>>>>> for (int i=0; i< 10; i++) >>>>>>> { >>>>>>> try { >>>>>>> result=client.scannerGet(aScannerId); >>>>>>> } catch (Exception ex1) >>>>>>> { >>>>>>> >>>>>>> } >>>>>>> if (result ==null) break; >>>>>>> >>>>>>> String >>>>>>> timeFormat=result.getFieldValue(0).toString(); >>>>>>> String >>>>>>> customerId=result.getFieldValue(1).toString(); >>>>>>> System.out.println(timeFormat + "\t" + customerId); >>>>>>> } >>>>>>> } catch (Exception ex) >>>>>>> { >>>>>>> ex.printStackTrace(); >>>>>>> } >>>>>>> finally { >>>>>>> if (aScannerId >0 ) client.scannerClose(aScannerId); >>>>>>> } >>>>>>> } catch (Exception exp) >>>>>>> { >>>>>>> exp.printStackTrace(); >>>>>>> } >>>>>>> finally{ >>>>>>> transport.close(); >>>>>>> } >>>>>>> >>>>>>> } >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Todd Lipcon >>>>>> Software Engineer, Cloudera >>>>>> >>>>>> >>>> >>>> >>>> -- >>>> Todd Lipcon >>>> Software Engineer, Cloudera >>>> >>> >>> >>> >>> -- >>> Alexey Kovyrin >>> http://kovyrin.net/ >>> >> > > > > -- > Alexey Kovyrin > http://kovyrin.net/ >