Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B87A5110DC for ; Thu, 31 Jul 2014 07:38:55 +0000 (UTC) Received: (qmail 18337 invoked by uid 500); 31 Jul 2014 07:38:51 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 18225 invoked by uid 500); 31 Jul 2014 07:38:51 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 18209 invoked by uid 99); 31 Jul 2014 07:38:51 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jul 2014 07:38:51 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of deepak8.kumar@citi.com designates 67.231.153.94 as permitted sender) Received: from [67.231.153.94] (HELO mx-b.mail.citi.com) (67.231.153.94) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jul 2014 07:38:46 +0000 Received: from pps.filterd (m0008247 [127.0.0.1]) by mx0b-00123c02.pphosted.com (8.14.5/8.14.5) with SMTP id s6V7UGoa026452 for ; Thu, 31 Jul 2014 07:38:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=citi.com; h=from : to : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=maila; bh=wJpvqd2sjiidnOpuJAaXzVYETzFjqQyfm5ROuK+VWbU=; b=mOQL11kpXV7X0P5/6aAiQ4B5I0CNi8UAbO1xXSsNZDxembMpDYyW29u4pT1lRsgFrZD0 CsRa7WyoAdoUf/VF56c99nLT0Jb8mHTTM6AbWlA3SNhkElMydKP030FqBdNoEecmGPDb ErofZ+1kCFR0mA5DOT3IizT85FUaD03LrVZ7Iig6f8Fj46WvNtZU+k1DtTsiAHmrFW6b KeSSFIIbgBYLoALOd2IcxZUfZOfQhKzgHPls91HitdPm4YegPv/oNyBFAyqkTBNbhL+K BDIv87NEQbTcXnRrkU+b8kj9bqA2zneO3EkN3PqZ7/xbEyPPL5jii18tRXY7pIFIsxs/ 9Q== Received: from mail.citigroup.com (smtpoutbound.citigroup.com [192.193.220.20]) by mx0b-00123c02.pphosted.com with ESMTP id 1mtbm0xkju-1 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT) for ; Thu, 31 Jul 2014 07:38:20 +0000 Received: from imbhub-gt35.nam.nsroot.net (namdlpdimpgt06.nam.nsroot.net [169.193.226.15]) by smtpinbound.citigroup.com (Sentrion-MTA-4.2.2/Sentrion-MTA-4.2.2) with ESMTP id s6V7cJep028913 for ; Thu, 31 Jul 2014 07:38:19 GMT Received: from EXTXIHT04.nam.nsroot.net (EXTXIHT04.nam.nsroot.net [169.177.87.31]) by imbhub-gt35.nam.nsroot.net (Sentrion-MTA-4.2.2/Sentrion-MTA-4.2.2) with ESMTP id s6V7cIMI010703 for ; Thu, 31 Jul 2014 07:38:19 GMT Received: from EXTXMB19.nam.nsroot.net ([169.254.4.171]) by EXTXIHT04.nam.nsroot.net ([169.177.87.31]) with mapi id 14.03.0174.001; Thu, 31 Jul 2014 02:38:18 -0500 From: "Kumar, Deepak8 " To: "'user@hadoop.apache.org'" Subject: RE: Hadoop Realtime Queries Thread-Topic: Hadoop Realtime Queries Thread-Index: Ac+skZTfD0CzkzdLTOeN4NJ3R1rhswAAGOGg Date: Thu, 31 Jul 2014 07:38:17 +0000 Message-ID: <1FA690FF1F80B649BA7B8DFACF30700655D2F914@EXTXMB19.nam.nsroot.net> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [169.177.87.240] Content-Type: multipart/alternative; boundary="_000_1FA690FF1F80B649BA7B8DFACF30700655D2F914EXTXMB19namnsro_" MIME-Version: 1.0 X-CFilter-Loop: Reflected X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.12.52,1.0.14,0.0.0000 definitions=2014-07-31_03:2014-07-30,2014-07-31,1970-01-01 signatures=0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_1FA690FF1F80B649BA7B8DFACF30700655D2F914EXTXMB19namnsro_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, As far as I know, real time queries are only possible using HBase & clouder= a search. Hive would be a batch process, it is not real time. So instead of= tuning different parameters , may be you could look for different architec= ture design so that you could use HBase. Regards, Deepak From: Natarajan, Prabakaran 1. (NSN - IN/Bangalore) [mailto:prabakaran.1.na= tarajan@nsn.com] Sent: Thursday, July 31, 2014 3:32 AM To: user@hadoop.apache.org Subject: Hadoop Realtime Queries Hi I want to perform realtime query on HDFS data. I tried hadoop/yarnt/hive,= shark on spark, Tez, etc., But still I couldn't get subsecond performance on the large data that I hav= e. I understand hadoop is not meant for this, but still want to achieve as max= as possible 1. How can we tune RHEL OS for this? 2. How can we tune yarn? 3. Is there is any stable framework like Tez which can perform much b= etter 4. Is there is any caching strategy that we can adopt? 5. Any articles related to this are welcome Thanks in Advance Prabakaran.N --_000_1FA690FF1F80B649BA7B8DFACF30700655D2F914EXTXMB19namnsro_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

As far as I know, real ti= me queries are only possible using HBase & cloudera search. Hive would = be a batch process, it is not real time. So instead of tuning different parameters , may be you could look for different architecture de= sign so that you could use HBase.

 <= /p>

Regards,

Deepak<= /p>

 <= /p>

From: Nataraja= n, Prabakaran 1. (NSN - IN/Bangalore) [mailto:prabakaran.1.natarajan@nsn.co= m]
Sent: Thursday, July 31, 2014 3:32 AM
To: user@hadoop.apache.org
Subject: Hadoop Realtime Queries

 

Hi

 

I want to perform realtime query on HDF= S data.   I tried hadoop/yarnt/hive, shark on spark, Tez, etc.,

But still I couldn’t get subsecon= d performance on the large data that I have. 

I understand hadoop is not meant for th= is, but still want to achieve as max as possible

 

1.    &nb= sp;  How can we tune RHEL OS for thi= s?

2.    &nb= sp;  How can we tune yarn?

3.    &nb= sp;  Is there is any stable framewor= k like Tez which can perform much better

4.    &nb= sp;  Is there is any caching strateg= y that we can adopt?

5.    &nb= sp;  Any articles related to this ar= e welcome

 

Thanks in Advance

 

Prabakaran.N

 

 

 

--_000_1FA690FF1F80B649BA7B8DFACF30700655D2F914EXTXMB19namnsro_--