hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <>
Subject Re: Mappers spawning Hive queries
Date Sat, 16 Apr 2016 07:10:13 GMT
Just out of curiosity, what is the use case behind this?

How do you call the shell script?

> On 16 Apr 2016, at 00:24, Shirish Tatikonda <> wrote:
> Hello,
> I am trying to run multiple hive queries in parallel by submitting them through a map-reduce
> More specifically, I have a map-only hadoop streaming job where each mapper runs a shell
script that does two things -- 1) parses input lines obtained via streaming; and 2) submits
a very simple hive query (via hive -e ...) with parameters computed from step-1. 
> Now, when I run the streaming job, the mappers seem to be stuck and I don't know what
is going on. When I looked on resource manager web UI, I don't see any new MR Jobs (triggered
from the hive query). I am trying to understand this behavior. 
> This may be a bad idea to begin with, and there may be better ways to accomplish the
same task. However, I would like to understand the behavior of such a MR job.
> Any thoughts?
> Thank you,
> Shirish

View raw message