Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EC142D720 for ; Thu, 13 Dec 2012 08:28:40 +0000 (UTC) Received: (qmail 1881 invoked by uid 500); 13 Dec 2012 08:28:39 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 1724 invoked by uid 500); 13 Dec 2012 08:28:38 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 1624 invoked by uid 99); 13 Dec 2012 08:28:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Dec 2012 08:28:37 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of chenbenhua@huawei.com designates 119.145.14.65 as permitted sender) Received: from [119.145.14.65] (HELO szxga02-in.huawei.com) (119.145.14.65) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Dec 2012 08:28:31 +0000 Received: from 172.24.2.119 (EHLO szxeml206-edg.china.huawei.com) ([172.24.2.119]) by szxrg02-dlp.huawei.com (MOS 4.3.4-GA FastPath queued) with ESMTP id AUD12683; Thu, 13 Dec 2012 16:28:09 +0800 (CST) Received: from SZXEML443-HUB.china.huawei.com (10.82.67.181) by szxeml206-edg.china.huawei.com (172.24.2.59) with Microsoft SMTP Server (TLS) id 14.1.323.3; Thu, 13 Dec 2012 16:27:57 +0800 Received: from SZXEML505-MBS.china.huawei.com ([169.254.2.68]) by SZXEML443-HUB.china.huawei.com ([10.82.67.181]) with mapi id 14.01.0323.003; Thu, 13 Dec 2012 16:27:51 +0800 From: Chenbenhua To: "user@hive.apache.org" Subject: =?gb2312?B?tPC4tDogUkVTVCBBUEkgZm9yIEhpdmUgcXVlcmllcz8=?= Thread-Topic: REST API for Hive queries? Thread-Index: AQHN2QoVJ7YucmraOEehJtx+1IrTFpgWYuYw Date: Thu, 13 Dec 2012 08:27:43 +0000 Message-ID: <112C82937743A34A82C33720ADFF81A0291CB1A9@SZXEML505-MBS.china.huawei.com> References: In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-cr-hashedpuzzle: BYKM C3hc DSC1 Dohx DsW1 D6N4 EKQl FWuo Gb8j G3xh JIbg Jbit JdQp J0td KEiQ KSz/;1;dQBzAGUAcgBAAGgAaQB2AGUALgBhAHAAYQBjAGgAZQAuAG8AcgBnAA==;Sosha1_v1;7;{86B2161D-43A1-4990-97C7-E2BAAE6B4624};YwBoAGUAbgBiAGUAbgBoAHUAYQBAAGgAdQBhAHcAZQBpAC4AYwBvAG0A;Thu, 13 Dec 2012 08:27:40 GMT;VHsNWToAIABSAEUAUwBUACAAQQBQAEkAIABmAG8AcgAgAEgAaQB2AGUAIABxAHUAZQByAGkAZQBzAD8A x-cr-puzzleid: {86B2161D-43A1-4990-97C7-E2BAAE6B4624} x-originating-ip: [10.66.110.77] Content-Type: multipart/alternative; boundary="_000_112C82937743A34A82C33720ADFF81A0291CB1A9SZXEML505MBSchi_" MIME-Version: 1.0 X-CFilter-Loop: Reflected X-Virus-Checked: Checked by ClamAV on apache.org --_000_112C82937743A34A82C33720ADFF81A0291CB1A9SZXEML505MBSchi_ Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: base64 SSBhZ3JlZSB3aXRoIE1hbmlzaCBNYWxob3RyYQ0KWW91IHNob3VsZCBwdXQgYW4gZXllIG9uIHRo ZSBhY2Nlc3Mgc3BlZWQgaWYgeW91IHdhbnQgZGlzcGxheSB0aGUgZGF0YSBvbiBhbiBVSSBhcHBs aWNhdGlvbg0KUHVzaCB0aGUgcmVzdWx0IGRhdGEgdG8gUkRCUyBtYXkgYmUgdGhlIGJlc3QgY2hv aWNlLg0KDQq3orz+yMs6IE1hbmlzaCBNYWxob3RyYSBbbWFpbHRvOm1hbmlzaC5oYWRvb3Aud29y a0BnbWFpbC5jb21dDQq3osvNyrG85DogMjAxMsTqMTLUwjEzyNUgMTY6MTUNCsrVvP7IyzogdXNl ckBoaXZlLmFwYWNoZS5vcmcNCtb3zOI6IFJlOiBSRVNUIEFQSSBmb3IgSGl2ZSBxdWVyaWVzPw0K DQoNCklkZWFsbHksIHB1c2ggdGhlIGFnZ3JlZ2F0ZWQgZGF0YSB0byBzb21lIFJEQk1TIGxpa2Ug TXlTUUwgYW5kIGhhdmUgUkVTVCBBUEkgb3Igc29tZSBBUEkgdG8gZW5hYmxlIHVpIHRvIGJ1aWxk IHJlcG9ydCBvciBxdWVyeSBvdXQgb2YgaXQuDQoNCklmIHRoZSB1c2UgY2FzZSBpcyBhZC1ob2Mg cXVlcnkgdGhlbiBvbmNlIHRoYXQgcXJ5IGlzIHN1Ym1pdHRlZCwgYW5kIHJlc3VsdCBpcyBnZW5l cmF0ZWQgaW4gYmF0Y2ggbW9kZSwgdGhlIFJFU1QgQVBJIGNhbiBiZSBwcm92aWRlZCB0byBnZXQg dGhlIHJlc3VsdHMgZnJvbSBIREZTIGRpcmVjdGx5Lg0KRm9yIHRoaXMgY2FuIHVzZSBXZWJIREZT IG9yIGJ1aWxkIG93biB3aGljaCBjYW4gaW50ZXJuYWxseSB1c2luZyBGaWxlU3lzdGVtIEFQSS4N Cg0KUmVnYXJkcywNCk1hbmlzaA0KDQpPbiBXZWQsIERlYyAxMiwgMjAxMiBhdCAxMTozMCBQTSwg Tml0aW4gUGF3YXIgPG5pdGlucGF3YXI0MzJAZ21haWwuY29tPG1haWx0bzpuaXRpbnBhd2FyNDMy QGdtYWlsLmNvbT4+IHdyb3RlOg0KSGl2ZSB0YWtlcyBhIGxvbmdlciB0aW1lIHRvIHJlc3BvbmQg dG8gcXVlcmllcyBhcyB0aGUgZGF0YSBnZXRzIGxhcmdlci4NCg0KQmVzdCB3YXkgdG8gaGFuZGxl IHRoaXMgaXMgeW91IHByb2Nlc3MgdGhlIGRhdGEgb24gaGl2ZSBhbmQgc3RvcmUgaW4gc29tZSBy ZGJtcyBsaWtlIG15c3FsIGV0Yy4NCk9uIHRvcCBvZiB0aGF0IHRoZW4geW91IGNhbiB3cml0ZSB5 b3VyIG93biBBUEkgb3IgdXNlIHBlbnRhaG8gbGlrZSBpbnRlcmZhY2Ugd2hlcmUgdGhleSBjYW4g d3JpdGUgdGhlIHF1ZXJpZXMgb3Igc2VlIHByZWRlZmluZWQgcmVwb3J0cy4NCg0KQWx0ZXJuYXRp dmVseSwgcGVudGFobyBkb2VzIGhhdmUgaGl2ZSBjb25uZWN0aW9uIGFzIHdlbGwuIFRoZXJlIGFy ZSBvdGhlciBwbGF0Zm9ybXMgc3VjaCBhcyB0YWxlbmQsIGRhdGFtZWVyIGV0Yy4gWW91IGNhbiBo YXZlIGEgbG9vayBhdCB0aGVtDQoNCk9uIFRodSwgRGVjIDEzLCAyMDEyIGF0IDE6MTUgQU0sIExl ZW5hIEd1cHRhIDxndXB0YS5sZWVuYUBnbWFpbC5jb208bWFpbHRvOmd1cHRhLmxlZW5hQGdtYWls LmNvbT4+IHdyb3RlOg0KSGksDQoNCldlIGFyZSB1c2luZyBIaXZlIGFzIG91ciBkYXRhIHdhcmVo b3VzZSB0byBydW4gdmFyaW91cyBxdWVyaWVzIG9uIGxhcmdlIGFtb3VudHMgb2YgZGF0YS4gVGhl cmUgYXJlIHNvbWUgdXNlcnMgd2hvIHdvdWxkIGxpa2UgdG8gZ2V0IGFjY2VzcyB0byB0aGUgb3V0 cHV0IG9mIHRoZXNlIHF1ZXJpZXMgYW5kIGRpc3BsYXkgdGhlIGRhdGEgb24gYW4gZXhpc3Rpbmcg VUkgYXBwbGljYXRpb24uIFdoYXQgaXMgdGhlIGJlc3Qgd2F5IHRvIGdpdmUgdGhlbSB0aGUgb3V0 cHV0IG9mIHRoZXNlIHF1ZXJpZXM/IFNob3VsZCB3ZSB3cml0ZSBSRVNUIEFQSXMgdGhhdCB0aGUg RnJvbnQgZW5kIGNhbiBjYWxsIHRvIGdldCB0aGUgZGF0YT8gSG93IGNhbiB0aGlzIGJlIGRvbmU/ DQogSSdkIGxpa2UgdG8ga25vdyB3aGF0IGhhdmUgb3RoZXIgcGVvcGxlIGRvbmUgdG8gbWVldCB0 aGlzIHJlcXVpcmVtZW50ID8NCkFueSBwb2ludGVycyB3b3VsZCBiZSB2ZXJ5IGhlbHBmdWwuDQpU aGFua3MuDQoNCg0KDQotLQ0KTml0aW4gUGF3YXINCg0K --_000_112C82937743A34A82C33720ADFF81A0291CB1A9SZXEML505MBSchi_ Content-Type: text/html; charset="gb2312" Content-Transfer-Encoding: quoted-printable

I agree with Manish Malhotra

You should put an eye on the access speed if you want = display the data on an UI application

Push the result data to RDBS may be the best choice.

 = ;

=B7=A2=BC=FE=C8=CB: Manish = Malhotra [mailto:manish.hadoop.work@gmail.com]
=B7=A2= =CB=CD=CA=B1=BC=E4: 2012=C4=EA12=D4=C213=C8=D5 16:15
=CA=D5=BC=FE=C8=CB: user@hive.apache.org
=D6=F7=CC=E2: Re: REST API for Hive queries?

 

 

Ideally, push the aggregated da= ta to some RDBMS like MySQL and have REST API or some API to enable ui to b= uild report or query out of it. 

 

If the use case is ad-hoc query= then once that qry is submitted, and result is generated in batch mode, th= e REST API can be provided to get the results from HDFS directly.

For this can use WebHDFS or bui= ld own which can internally using FileSystem API. 

 

Regards,

Manish

=  

On Wed, Dec 12, 2012 at 11:30 P= M, Nitin Pawar <nitinpawar432@gmail.com> wrote:

Hive takes a longer time to res= pond to queries as the data gets larger.

 

Best way to handle this is you = process the data on hive and store in some rdbms like mysql etc.=

On top of that then you can wri= te your own API or use pentaho like interface where they can write the quer= ies or see predefined reports. 

 

Alternatively, pentaho does hav= e hive connection as well. There are other platforms such as talend, datame= er etc. You can have a look at them 

=  

On Thu, Dec 13, 2012 at 1:15 AM= , Leena Gupta <gupta.leena@gmail.com> wrote:

Hi,

 

We are using Hive as our data w= arehouse to run various queries on large amounts of data. There are some us= ers who would like to get access to the output of these queries and display= the data on an existing UI application. What is the best way to give them the output of these queries? Should we w= rite REST APIs that the Front end can call to get the data? How can this be= done? 

 I'd like to know what hav= e other people done to meet this requirement ?

Any pointers would be very help= ful.

Thanks.



 

--
Nitin Pawar<= /o:p>

 

--_000_112C82937743A34A82C33720ADFF81A0291CB1A9SZXEML505MBSchi_--