hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Wiley <kwi...@keithwiley.com>
Subject Hive performance vs. SQL?
Date Mon, 19 Mar 2012 22:51:41 GMT
I haven't had an opportunity to set up a huge Hive database yet because exporting csv files
from our SQL database is, in itself, a rather laborious task.  I was just curious how I might
expect Hive to perform vs. SQL on large databases and large queries?  I realize Hive is pretty
"latent" since it builds and runs MapReduce jobs for even the simplest queries, but that is
precisely why I think it might perform better on long queries against large (external CSV)
databases).

Would you expect Hive to ever outperform SQL on a single machine (standalone or pseudo-distributed
mode)?  I am entirely open to the possibility that the answer is no, that Hive could never
compete with SQL in a single machine.  Is this true?

If so, how large (how parallel) do you think the underlying Hadoop cluster needs to be before
Hive overtakes SQL?  2X?  10X?  Where is the crossover point where Hive actually outperforms
SQL?

Along similar lines, might Hive never outperform SQL on a database small enough for SQL to
run on a single machine, a 10s to 100s of GBs?  Must the database itself be so large that
SQL is effectively crippled and the data must be distributed before Hive offer significant
gains?

I am really just trying to get a basic feel for how I might anticipate's Hive's behavior vs.
SQL once I get a large system up and running.

Thanks.

________________________________________________________________________________
Keith Wiley     kwiley@keithwiley.com     keithwiley.com    music.keithwiley.com

"I used to be with it, but then they changed what it was.  Now, what I'm with
isn't it, and what's it seems weird and scary to me."
                                           --  Abe (Grandpa) Simpson
________________________________________________________________________________


Mime
View raw message