hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Douglas Moore <douglas.mo...@thinkbiganalytics.com>
Subject Hive 0.13 - Yarn - Tez application not shutting down
Date Thu, 12 Jun 2014 20:15:25 GMT
I'm on HDP 2.1 Hive 0.13/Tez

I run this query:

hive -v --database=ORC_db -e `select * from store_sales where
ss_ticket_number=5741230 and ss_item_sk=4825`

The yarn application runs Tez for a few seconds, generates the results and
shuts down and goes away as reported by `yarn application -list`. All is
well.

I run the query below, Tez starts up, runs for a few seconds, generates the
same results, Tez session shuts down and then the yarn application still
shows up as 'RUNNING' minutes later.
Why? How do I fix this 'leak' or re-use the YARN application?
The problem is after running 30 or so of these type of scripts via hive,
the jobs stop running and 'hang'

Thanks!

hive -v --database=ORC_db -f
set hive.optimize.ppd=true
set hive.exec.parallel=true
set hive.vectorized.execution.enabled=true
set io.sort.mb=512
set hive.exec.reducers.bytes.per.reducer=134217728
set hive.auto.convert.join.noconditionaltask.size=1200000000
set mapreduce.job.reduce.slowstart.completedmaps=0.5
set hive.mapjoin.localtask.max.memory.usage=0.99

select * from store_sales where ss_ticket_number=5741230 and ss_item_sk=4825

Mime
View raw message