hawq-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leon Zhang <leonca...@gmail.com>
Subject Problem about Hawq on YARN
Date Fri, 27 Nov 2015 07:23:59 GMT
Hi, HAWQ Developers:

     Thanks for the friendly hawq community, they help me to setup the
hawq-2.0 on YARN. After I execute the TPC-DS queries, I encounter some yarn
error for part of queries.

     For instance, the query1:
-- start query 12 in stream 0 using template query1.tpl and seed 1711160884
with customer_total_return as
(select sr_customer_sk as ctr_customer_sk
,sr_store_sk as ctr_store_sk
,sum(SR_REVERSED_CHARGE) as ctr_total_return
from store_returns
,date_dim
where sr_returned_date_sk = d_date_sk
and d_year =1998
group by sr_customer_sk
,sr_store_sk)
 select  c_customer_id
from customer_total_return ctr1
,store
,customer
where ctr1.ctr_total_return > (select avg(ctr_total_return)*1.2
from customer_total_return ctr2
where ctr1.ctr_store_sk = ctr2.ctr_store_sk)
and s_store_sk = ctr1.ctr_store_sk
and s_state = 'TN'
and ctr1.ctr_customer_sk = c_customer_sk
order by c_customer_id
limit 100;
psql:/mnt/xiaolin/query/scale_1/query_0.sql:502: WARNING:  Sync RPC
framework (inet) finds exception raised.
psql:/mnt/xiaolin/query/scale_1/query_0.sql:502: ERROR:  failed to return
resource to HAWQ resource manager because of failing to receive content.
(pquery.c:983)

     And many other errors like this:
$ grep psql:/mnt/xiaolin/query/scale_1/query_0.sql
hawq_yarn/log_1/log_1.log
psql:/mnt/xiaolin/query/scale_1/query_0.sql:502: WARNING:  Sync RPC
framework (inet) finds exception raised.
psql:/mnt/xiaolin/query/scale_1/query_0.sql:502: ERROR:  failed to return
resource to HAWQ resource manager because of failing to receive content.
(pquery.c:983)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:1103: WARNING:  Sync RPC
framework (inet) finds exception raised.
psql:/mnt/xiaolin/query/scale_1/query_0.sql:1103: ERROR:  failed to return
resource to HAWQ resource manager because of failing to receive content.
(pquery.c:983)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:1132: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:1997: WARNING:  Sync RPC
framework (inet) finds exception raised.
psql:/mnt/xiaolin/query/scale_1/query_0.sql:1997: ERROR:  failed to return
resource to HAWQ resource manager because of failing to receive content.
(pquery.c:983)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2150: WARNING:  Sync RPC
framework (inet) finds exception raised.
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2150: ERROR:  failed to return
resource to HAWQ resource manager because of failing to receive content.
(pquery.c:983)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2455: ERROR:  insufficient
memory reserved for statement (execHHashagg.c:732)  (seg12 dserver2:40404
pid=331172) (dispatcher.c:1701)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2543: WARNING:  Sync RPC
framework (inet) finds exception raised.
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2543: ERROR:  failed to return
resource to HAWQ resource manager because of failing to receive content.
(pquery.c:983)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2639: WARNING:  Sync RPC
framework (inet) finds exception raised.
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2639: ERROR:  failed to return
resource to HAWQ resource manager because of failing to receive content.
(pquery.c:983)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2661: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2720: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2742: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2794: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2902: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:2935: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3182: WARNING:  Sync RPC
framework (inet) finds exception raised.
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3182: ERROR:  failed to return
resource to HAWQ resource manager because of failing to receive content.
(pquery.c:983)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3286: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3380: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3428: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3457: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3497: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3528: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3595: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3626: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3687: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3710: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3817: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3845: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3866: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3922: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:3936: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:4004: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:4064: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:4085: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:4113: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:4489: ERROR:  insufficient
memory reserved for statement
psql:/mnt/xiaolin/query/scale_1/query_0.sql:4633: WARNING:  Sync RPC
framework (inet) finds exception raised.
psql:/mnt/xiaolin/query/scale_1/query_0.sql:4633: ERROR:  failed to return
resource to HAWQ resource manager because of failing to receive content.
(pquery.c:983)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:4677: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)
psql:/mnt/xiaolin/query/scale_1/query_0.sql:4805: ERROR:  failed to acquire
resource because of too many unavailable segments. (pquery.c:798)

    I read through the log of YARN, and I can not find any error about
this. I am not sure if any parameters can be tuned to fix issue? Or if it
is a bug?

    Any help will be appreciated, thanks in advance.

Mime
View raw message