hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Wong <sw...@netflix.com>
Subject RE:
Date Thu, 12 Jan 2012 01:41:58 GMT
Try fetchOne or fetchN.

From: Lu, Wei [mailto:wlu@microstrategy.com]
Sent: Tuesday, January 10, 2012 11:14 PM
To: Lu, Wei; user@hive.apache.org
Subject: RE:

BTW, I am using hive 0.7

From: Lu, Wei
Sent: Wednesday, January 11, 2012 3:13 PM
To: 'user@hive.apache.org'
Subject:

Hi,

I am using ThriftHive.Client to access a pretty large table.

SQL Statement:
select    a11.asin  asin,
                max(a11.title)  title,
                a11.salesrank  salesrank,
                a11.category  category,
                avg(a11.avg_rating)  WJXBFS1,
                sum(a11.total_num_reviews)  WJXBFS2,
                sum(a11.num_subcategories)  WJXBFS3
from      table_details      a11
group by              a11.asin,
                a11.salesrank,
                a11.category.

The statement will select a pretty large result set (1,000,000+ rows). When I use ThriftHive.Client
fetchAll() to get all the row strings, an exception returns like below:

Exception in thread "main" org.apache.thrift.TApplicationException: Internal error processing
fetchAll
      at org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
      at org.apache.hadoop.hive.service.ThriftHive$Client.recv_fetchAll(ThriftHive.java:224)
      at org.apache.hadoop.hive.service.ThriftHive$Client.fetchAll(ThriftHive.java:208)
      ... ...

Why does that happen? How could I deal with it? It seem that incremental fetch is not supported
by hive.

Regards,
Wei

Mime
View raw message