hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wen Lin <w...@pivotal.io>
Subject Re: HAWQ-1548 - Ambiguous message while logging hawq utilization
Date Tue, 14 Nov 2017 03:40:12 GMT
Hi, Shubham,

The second message means that, this YARN cluster's total resource is 1376256
MB, 168.000000 CORE.
And HAWQ clusters have the resource: 98304MB, 12.000000 CORE of the whole
YARN cluster now.
The other resources in YARN haven't been allocated for HAWQ.
Maybe there are no HAWQ segments running on these segments at the moment,
or other reasons.

Thanks!


On Tue, Nov 14, 2017 at 9:25 AM, Shubham Sharma <ssharma@pivotal.io> wrote:

> While YARN mode is enabled, resource broker logs two things -
>
>    - YARN cluster total resource
>    - HAWQ's total resource per node.
>
> Following messages are logged
>
> 2017-11-11 23:21:40.944904
> UTC,,,p549330,th900077856,,,,0,con4,,seg-10000,,,,,"LOG","00000","Resource
> manager YARN resource broker counted YARN cluster having total
> resource (1376256 MB, 168.000000
> CORE).",,,,,,,0,,"resourcebroker_LIBYARN.c",776,
>
> 2017-11-11 23:21:40.944921
> UTC,,,p549330,th900077856,,,,0,con4,,seg-10000,,,,,"LOG","00000","Resource
> manager YARN resource broker counted HAWQ cluster now having (98304
> MB, 12.000000 CORE) in a YARN cluster of total resource (1376256 MB,
> 168.000000 CORE).",,,,,,,0,,"resourcebroker_LIBYARN.c",785,
>
> The second message shown above is ambiguous, After reading the sentence
> below it looks like that complete Hawq cluster in whole has only 98304 MB
> and 12 cores. However according to the configuration it should be 98304 MB
> and 12 cores per segment server.
>
> Resource manager YARN resource broker counted HAWQ cluster now having
> (98304 MB, 12.000000 CORE) in a YARN cluster of total resource
> (1376256 MB, 168.000000 CORE).
>
> Either the wrong variables are printed or we can correct the message to
> represent that the resources logged are per node. As this can confuse the
> user into thinking that hawq cluster does not have enough resources.
>
> Is this understanding correct ?
>
> Regards,
> Shubham Sharma
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message