hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xuefu Zhang <xzh...@cloudera.com>
Subject Re: Hive on Spark for simple hive queries
Date Tue, 04 Nov 2014 18:20:54 GMT
For certain simple queries like your example, Hive doesn't some
optimization by execute it locally, which means no jobs are submitted to
the cluster (MR or Spark). I'm not sure if there is a way to turn this off,
but this is true for all execution engines.

Thanks,
Xuefu

On Tue, Nov 4, 2014 at 8:36 AM, Prabu Soundar Rajan -X (prabsoun - MINDTREE
LIMITED at Cisco) <prabsoun@cisco.com> wrote:

> Hi Team,
>
> In spite of setting hive execution engine as Spark, when we try simple
> hive queries having only mapper phase like (select * from table where
> column=xyz) - we observe the jobs are not submitted to Spark master. We do
> not see those jobs in Spark master web UI. But when we try some queries
> with reducer phase(in mr execution style), we see the job as "Hive on
> Spark" application in Spark master web UI. Appreciate if you could help us
> understand this behavior.   Am I missing something obvious here?
>
> Thanks & Regards,
> Prabu
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message