hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Kovyrin <ale...@kovyrin.net>
Subject Re: thrift for hbase in CDH3 broken ?
Date Fri, 03 Sep 2010 07:31:13 GMT
yes, Centos 5.5 + CDH3b2

On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <jinsong_hu@hotmail.com> wrote:
> are you using CDH3 distribution ?
>
> Jinsong
>
>
> --------------------------------------------------
> From: "Alexey Kovyrin" <alexey@kovyrin.net>
> Sent: Friday, September 03, 2010 12:04 AM
> To: <user@hbase.apache.org>
> Subject: Re: thrift for hbase in CDH3 broken ?
>
>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>> this example and tested it in our cluster, works as expected.
>> For this to work you'd need to install rubygems and thrift gem (gem
>> install thrift).
>>
>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <jinsong_hu@hotmail.com>
>> wrote:
>>>
>>> Can you send me some ruby test code and so I can try against the latest
>>> CDH3
>>> ?
>>>
>>> Jimmy.
>>>
>>> --------------------------------------------------
>>> From: "Alexey Kovyrin" <alexey@kovyrin.net>
>>> Sent: Thursday, September 02, 2010 8:15 PM
>>> To: <user@hbase.apache.org>
>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>
>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>
>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <todd@cloudera.com> wrote:
>>>>>
>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <jinsong_hu@hotmail.com>
>>>>> wrote:
>>>>>
>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>
>>>>>> and the fact that the API
>>>>>>
>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>
>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>        {
>>>>>>>>            System.out.println(new String(name));
>>>>>>>>        }
>>>>>>>>
>>>>>>>
>>>>>> successfully printed all table names shows that it is indeed thrift
>>>>>> server.
>>>>>>
>>>>>> if it is hue, it won't print the table names.
>>>>>>
>>>>>> Ah, sorry, I missed that in your original message. Not sure what's
up,
>>>>>> then
>>>>>
>>>>> - we don't have any changes in CDH that would affect this. Anyone here
>>>>> used
>>>>> thrift on 0.89.20100621?
>>>>>
>>>>> -Todd
>>>>>
>>>>>
>>>>>
>>>>>> Jimmy.
>>>>>>
>>>>>> --------------------------------------------------
>>>>>> From: "Todd Lipcon" <todd@cloudera.com>
>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>
>>>>>> To: <user@hbase.apache.org>
>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>
>>>>>>
>>>>>>  Hi Jinsong,
>>>>>>>
>>>>>>> Are you sure that the port you're connecting to is indeed the
thrift
>>>>>>> server?
>>>>>>>
>>>>>>> Unfortunately both the HBase thrift server and the Hue namenode
>>>>>>> plugin
>>>>>>> listen on port 9090, so you might be having an issue where your
HBase
>>>>>>> client
>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>
>>>>>>> You can verify the ports using a command like "/sbin/fuser -n
tcp
>>>>>>> 9090"
>>>>>>> to
>>>>>>> see which pid has it open, then cross reference against sudo
jps.
>>>>>>>
>>>>>>> Thanks
>>>>>>> -Todd
>>>>>>>
>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <jinsong_hu@hotmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>  Hi, There,
>>>>>>>>
>>>>>>>>  I am trying to test and see if thrift for hbase works.
I followed
>>>>>>>> the
>>>>>>>> example from
>>>>>>>>
>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>
>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>> returns all table names successfully, but the scanner never
returned
>>>>>>>> any record. and it even throws exception:
>>>>>>>>
>>>>>>>> org.apache.thrift.TApplicationException: get failed: unknown
result
>>>>>>>>     at
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>>     at
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>>     at
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>>     at
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>>
>>>>>>>> I checked the syntax and the table and confirmed that the
table does
>>>>>>>> have
>>>>>>>> record and it should
>>>>>>>> return result, but it doesn't. Can anybody tell me what is
wrong ?
>>>>>>>>
>>>>>>>> At this time, I am suspecting the thrift server shipped with
CDH3
>>>>>>>> may
>>>>>>>> have
>>>>>>>> problem. if there anyway
>>>>>>>> I can get this test code working ?
>>>>>>>>
>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found
that the
>>>>>>>> code
>>>>>>>> generated from hbase-thrift project
>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>
>>>>>>>>
>>>>>>>> Jimmy.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>  public void testThriftAPI()
>>>>>>>>  {
>>>>>>>>    // Make socket
>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>    // Wrap in a protocol
>>>>>>>>    TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>>
>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>
>>>>>>>>    try {
>>>>>>>>        transport.open();
>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>        {
>>>>>>>>            System.out.println(new String(name));
>>>>>>>>        }
>>>>>>>>
>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>        String startRow="";
>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>
>>>>>>>>        int aScannerId=0;
>>>>>>>>        try {
>>>>>>>>
>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>            if (cell != null)
>>>>>>>>            {
>>>>>>>>               String value= new String( cell.value);
>>>>>>>>               System.out.println(value);
>>>>>>>>            }
>>>>>>>>            aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>            TRowResult result=null;
>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>            {
>>>>>>>>                try {
>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>                } catch (Exception ex1)
>>>>>>>>                {
>>>>>>>>
>>>>>>>>                }
>>>>>>>>                if (result ==null) break;
>>>>>>>>
>>>>>>>>                String timeFormat=result.getFieldValue(0).toString();
>>>>>>>>                String customerId=result.getFieldValue(1).toString();
>>>>>>>>                System.out.println(timeFormat + "\t"
+ customerId);
>>>>>>>>            }
>>>>>>>>        } catch (Exception ex)
>>>>>>>>        {
>>>>>>>>            ex.printStackTrace();
>>>>>>>>        }
>>>>>>>>        finally {
>>>>>>>>            if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>>>>        }
>>>>>>>>    } catch (Exception exp)
>>>>>>>>    {
>>>>>>>>        exp.printStackTrace();
>>>>>>>>    }
>>>>>>>>    finally{
>>>>>>>>        transport.close();
>>>>>>>>    }
>>>>>>>>
>>>>>>>>  }
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Todd Lipcon
>>>>>>> Software Engineer, Cloudera
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Todd Lipcon
>>>>> Software Engineer, Cloudera
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Alexey Kovyrin
>>>> http://kovyrin.net/
>>>>
>>>
>>
>>
>>
>> --
>> Alexey Kovyrin
>> http://kovyrin.net/
>>
>



-- 
Alexey Kovyrin
http://kovyrin.net/

Mime
View raw message