hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Natarajan, Prabakaran 1. (NSN - IN/Bangalore)" <prabakaran.1.natara...@nsn.com>
Subject RE: Hadoop Realtime Queries
Date Thu, 31 Jul 2014 16:07:50 GMT
Hi Nitin,

I want queries to return within a second

Hive table DataSize is 50TB – Snappy RC file

Thanks and Regards
Prabakaran.N  aka NP
nsn, Bangalore
When "I" is replaced by "We" - even Illness becomes "Wellness"

From: ext Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Thursday, July 31, 2014 6:25 PM
To: user@hadoop.apache.org
Subject: Re: Hadoop Realtime Queries

I want quick response for SQL queries .

how quick is quick for you ?
what's the data size?
what kind of queries you want to run?
what is the frequency of running the query on same dataset again and again?

On Thu, Jul 31, 2014 at 6:20 PM, Natarajan, Prabakaran 1. (NSN - IN/Bangalore) <prabakaran.1.natarajan@nsn.com<mailto:prabakaran.1.natarajan@nsn.com>>

Thank you all for the reply.

I want quick response for SQL queries .

Thanks and Regards

From: ext Bertrand Dechoux [mailto:dechouxb@gmail.com<mailto:dechouxb@gmail.com>]
Sent: Thursday, July 31, 2014 1:28 PM
To: user@hadoop.apache.org<mailto:user@hadoop.apache.org>
Subject: Re: Hadoop Realtime Queries

It all depends on the context and what is really meant by realtime. Impala (and other concurrent
alternatives) are not listed among the tools you have tried.
Maybe you should not focus only on batch frameworks for providing a realtime access? The results
are not surprising.

Bertrand Dechoux

On Thu, Jul 31, 2014 at 9:38 AM, Kumar, Deepak8 <deepak8.kumar@citi.com<mailto:deepak8.kumar@citi.com>>
As far as I know, real time queries are only possible using HBase & cloudera search. Hive
would be a batch process, it is not real time. So instead of tuning different parameters ,
may be you could look for different architecture design so that you could use HBase.


From: Natarajan, Prabakaran 1. (NSN - IN/Bangalore) [mailto:prabakaran.1.natarajan@nsn.com<mailto:prabakaran.1.natarajan@nsn.com>]
Sent: Thursday, July 31, 2014 3:32 AM
To: user@hadoop.apache.org<mailto:user@hadoop.apache.org>
Subject: Hadoop Realtime Queries


I want to perform realtime query on HDFS data.   I tried hadoop/yarnt/hive, shark on spark,
Tez, etc.,
But still I couldn’t get subsecond performance on the large data that I have.
I understand hadoop is not meant for this, but still want to achieve as max as possible

1.       How can we tune RHEL OS for this?
2.       How can we tune yarn?
3.       Is there is any stable framework like Tez which can perform much better
4.       Is there is any caching strategy that we can adopt?
5.       Any articles related to this are welcome

Thanks in Advance


Nitin Pawar
View raw message