hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jinsong Hu" <jinsong...@hotmail.com>
Subject Re: thrift for hbase in CDH3 broken ?
Date Fri, 03 Sep 2010 18:17:03 GMT

I tried your code and indeed it works. but the java version doesn't work. so 
it looks like it is a bug
of the java library supplied by the thrift-0.2.0 version.

Jimmy.
--------------------------------------------------
From: "Alexey Kovyrin" <alexey@kovyrin.net>
Sent: Friday, September 03, 2010 12:31 AM
To: <user@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

> yes, Centos 5.5 + CDH3b2
>
> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <jinsong_hu@hotmail.com> wrote:
>> are you using CDH3 distribution ?
>>
>> Jinsong
>>
>>
>> --------------------------------------------------
>> From: "Alexey Kovyrin" <alexey@kovyrin.net>
>> Sent: Friday, September 03, 2010 12:04 AM
>> To: <user@hbase.apache.org>
>> Subject: Re: thrift for hbase in CDH3 broken ?
>>
>>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>>> this example and tested it in our cluster, works as expected.
>>> For this to work you'd need to install rubygems and thrift gem (gem
>>> install thrift).
>>>
>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <jinsong_hu@hotmail.com>
>>> wrote:
>>>>
>>>> Can you send me some ruby test code and so I can try against the latest
>>>> CDH3
>>>> ?
>>>>
>>>> Jimmy.
>>>>
>>>> --------------------------------------------------
>>>> From: "Alexey Kovyrin" <alexey@kovyrin.net>
>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>> To: <user@hbase.apache.org>
>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>
>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>
>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <todd@cloudera.com>

>>>>> wrote:
>>>>>>
>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <jinsong_hu@hotmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>
>>>>>>> and the fact that the API
>>>>>>>
>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>
>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>        {
>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>        }
>>>>>>>>>
>>>>>>>>
>>>>>>> successfully printed all table names shows that it is indeed
thrift
>>>>>>> server.
>>>>>>>
>>>>>>> if it is hue, it won't print the table names.
>>>>>>>
>>>>>>> Ah, sorry, I missed that in your original message. Not sure what's

>>>>>>> up,
>>>>>>> then
>>>>>>
>>>>>> - we don't have any changes in CDH that would affect this. Anyone

>>>>>> here
>>>>>> used
>>>>>> thrift on 0.89.20100621?
>>>>>>
>>>>>> -Todd
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Jimmy.
>>>>>>>
>>>>>>> --------------------------------------------------
>>>>>>> From: "Todd Lipcon" <todd@cloudera.com>
>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>
>>>>>>> To: <user@hbase.apache.org>
>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>
>>>>>>>
>>>>>>>  Hi Jinsong,
>>>>>>>>
>>>>>>>> Are you sure that the port you're connecting to is indeed
the 
>>>>>>>> thrift
>>>>>>>> server?
>>>>>>>>
>>>>>>>> Unfortunately both the HBase thrift server and the Hue namenode
>>>>>>>> plugin
>>>>>>>> listen on port 9090, so you might be having an issue where
your 
>>>>>>>> HBase
>>>>>>>> client
>>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>>
>>>>>>>> You can verify the ports using a command like "/sbin/fuser
-n tcp
>>>>>>>> 9090"
>>>>>>>> to
>>>>>>>> see which pid has it open, then cross reference against sudo
jps.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> -Todd
>>>>>>>>
>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <jinsong_hu@hotmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>  Hi, There,
>>>>>>>>>
>>>>>>>>>  I am trying to test and see if thrift for hbase works.
I followed
>>>>>>>>> the
>>>>>>>>> example from
>>>>>>>>>
>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>
>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>> returns all table names successfully, but the scanner
never 
>>>>>>>>> returned
>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>
>>>>>>>>> org.apache.thrift.TApplicationException: get failed:
unknown 
>>>>>>>>> result
>>>>>>>>>     at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>>>     at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>>>     at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>>>     at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>>>
>>>>>>>>> I checked the syntax and the table and confirmed that
the table 
>>>>>>>>> does
>>>>>>>>> have
>>>>>>>>> record and it should
>>>>>>>>> return result, but it doesn't. Can anybody tell me what
is wrong ?
>>>>>>>>>
>>>>>>>>> At this time, I am suspecting the thrift server shipped
with CDH3
>>>>>>>>> may
>>>>>>>>> have
>>>>>>>>> problem. if there anyway
>>>>>>>>> I can get this test code working ?
>>>>>>>>>
>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found
that the
>>>>>>>>> code
>>>>>>>>> generated from hbase-thrift project
>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Jimmy.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>  {
>>>>>>>>>    // Make socket
>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>    TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>>>
>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>
>>>>>>>>>    try {
>>>>>>>>>        transport.open();
>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>        {
>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>        }
>>>>>>>>>
>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>        String startRow="";
>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>
>>>>>>>>>        int aScannerId=0;
>>>>>>>>>        try {
>>>>>>>>>
>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>            if (cell != null)
>>>>>>>>>            {
>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>               System.out.println(value);
>>>>>>>>>            }
>>>>>>>>>            aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>            TRowResult result=null;
>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>            {
>>>>>>>>>                try {
>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>                {
>>>>>>>>>
>>>>>>>>>                }
>>>>>>>>>                if (result ==null) break;
>>>>>>>>>
>>>>>>>>>                String 
>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>                String 
>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>                System.out.println(timeFormat + "\t" +
customerId);
>>>>>>>>>            }
>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>        {
>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>        }
>>>>>>>>>        finally {
>>>>>>>>>            if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>>>>>        }
>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>    {
>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>    }
>>>>>>>>>    finally{
>>>>>>>>>        transport.close();
>>>>>>>>>    }
>>>>>>>>>
>>>>>>>>>  }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Todd Lipcon
>>>>>>>> Software Engineer, Cloudera
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Todd Lipcon
>>>>>> Software Engineer, Cloudera
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Alexey Kovyrin
>>>>> http://kovyrin.net/
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Alexey Kovyrin
>>> http://kovyrin.net/
>>>
>>
>
>
>
> -- 
> Alexey Kovyrin
> http://kovyrin.net/
> 

Mime
View raw message