Mailing-List: contact dev-help@hawq.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hawq.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <CAH=zMbpX=Xek8id2XgER=TYYiJH6oLibpTk7z+zR4+u7nBw=_A@mail.gmail.com>
References: <CAH_aXGMyWw=ZVUx2opZA+wfgMk6mf6WBhqarEHv8ObT=3RZevA@mail.gmail.com>
 <CABQrizc8SBfcnvYNP3+5uxaxbRSyG8mB_ZvnXZ_0k6YL_tOVhg@mail.gmail.com>
 <CAMjXoANyNw406aiD0yDO8OaN36mgQO=2E3uPLffw6v-FvsHDdw@mail.gmail.com>
 <CAH_aXGMaNhTOHUJ-zTjnt-jVKjba=RBBmtdbtXj6it6g+TC6nA@mail.gmail.com>
 <CAMjXoAObWvLjjHhnXSMzHyytC06Hz+tNNQON7gBiYbyEuxw=sA@mail.gmail.com>
 <CA+F1ufkCVBpwCiNu7u0JKiDxreRd-0cWaQkUD2vkCTBeFK6qOg@mail.gmail.com> <CAH=zMbpX=Xek8id2XgER=TYYiJH6oLibpTk7z+zR4+u7nBw=_A@mail.gmail.com>
From: Ming Li <mli@pivotal.io>
Date: Wed, 21 Dec 2016 14:58:13 +0800
Message-ID: <CA+F1uf=rXjgNuk=XJA+=wfypT48_nMuXuDja+TKuuqniqhMx4g@mail.gmail.com>
Subject: Re: unrecognized configuration parameter "ONETARY"
To: Radar Da lei <rlei@pivotal.io>
Cc: dev@hawq.incubator.apache.org
Content-Type: multipart/alternative; boundary=001a113ed15c346843054425ad3a
archived-at: Wed, 21 Dec 2016 06:58:20 -0000

--001a113ed15c346843054425ad3a
Content-Type: text/plain; charset=UTF-8

Thanks Radar,

It is strange that 'unrecognized configuration' will issue FATAL error, I
config in hawq-site.xml, and grep pg_log on master and segment date
directory, no FATAL error of 'unrecognized configuration'. Any ideas?

On Wed, Dec 21, 2016 at 2:45 PM, Radar Da lei <rlei@pivotal.io> wrote:

> Hi Ming,
>
> 'hawq_lc_monetary' is only used to store user defined '--lc-monetary' value
> then pass below option to initdb:
>       '--lc-monetary=${hawq_lc_monetary} '
>
> So finally we give '--lc-monetary=en_US.utf8 ' to initdb. Base on this I
> don't think the GUC 'ONETARY' have any dependencies with hawq admin
> tools. Thanks.
>
> Regards,
> Radar
>
> On Tue, Dec 20, 2016 at 10:18 AM, Ming Li <mli@pivotal.io> wrote:
>
>> Hi Radar,
>>
>> It seems that the config param 'hawq_lc_monetary' can only be recognized
>> by hawq admin tools in python, and set hawq guc 'lc-monetary' finally.
>>
>> My question is: Why not directly using 'lc-monetary' instead of '
>> hawq_lc_monetary' in the hawq admin tools? Is there any possible to
>> conflict?
>>
>> Thanks.
>>
>> On Tue, Dec 20, 2016 at 6:45 AM, Yi Jin <yjin@pivotal.io> wrote:
>>
>>> Thank you jon, I will specially recheck the guc population logic. Yi
>>>
>>> On Tue, Dec 20, 2016 at 4:07 AM, Jon Roberts <jroberts@pivotal.io>
>>> wrote:
>>>
>>> > The segment actually has this error:
>>> > 2016-12-14 13:47:34.760839
>>> > UTC,"gpadmin","gpadmin",p737499,th542214432,"172.21.
>>> > 13.196","40327",2016-12-14
>>> > 13:47:34 UTC,0,con23798,,seg-10000,,,,,"FATAL","42704","unrecognized
>>> > configuration parameter ""ONETARY""",,,,,,,0,,"guc.c",10006,
>>> >
>>> > This made me check out the configs because the error was raised from
>>> > guc.c.  I found that /usr/local/hawq/etc/_mgmt_config has "onetary"
>>> text:
>>> > hawq_lc_monetary=en_US.utf8
>>> >
>>> > I'm using nodes with 24 drives so I have 24 temp directories set for
>>> both
>>> > the master and the segments.  Thinking that possibly the problem is
>>> related
>>> > to the number of vSegs * the size of the config file exceeding a
>>> variable
>>> > size, I decided to reduce the temp directories down to just two for
>>> both
>>> > the master and segments.
>>> >
>>> > After restarting HAWQ, I could get the query to use 16 and even 24
>>> vSegs
>>> > without a problem.
>>> >
>>> > So maybe the guc logic needs revisiting to make sure there isn't an
>>> issue
>>> > with parsing the GUCs when there are many vSegs and many temp
>>> directories.
>>> > I would think this would be a problem even with the default number of
>>> vSegs
>>> > per host on really large clusters.
>>> >
>>> >
>>> > Jon Roberts
>>> > Principal Engineer | jroberts@pivotal.io | 615-426-8661
>>> <(615)%20426-8661>
>>> >
>>> > On Mon, Dec 19, 2016 at 4:44 AM, Yi Jin <yjin@pivotal.io> wrote:
>>> >
>>> > > Hi Jon,
>>> > >
>>> > > I think there is no ONTARY configuration item, so maybe I need the
>>> full
>>> > log
>>> > > containing that error to check the error routine.
>>> > >
>>> > > Best,
>>> > > Yi
>>> > >
>>> > > On Mon, Dec 19, 2016 at 5:39 PM, Paul Guo <paulguo@gmail.com> wrote:
>>> > >
>>> > > > You could grep the log to see whether there is "ONETARY" setting
>>> before
>>> > > > this error occurs, and also grep configuration files and related
>>> test
>>> > > files
>>> > > > to find who tried to set this.
>>> > > >
>>> > > > 2016-12-14 22:36 GMT+08:00 Jon Roberts <jroberts@pivotal.io>:
>>> > > >
>>> > > > > I'm getting the error message: unrecognized configuration
>>> parameter
>>> > > > > "ONETARY" with a few of the TPC-DS queries where I'm increasing
>>> the
>>> > > > number
>>> > > > > of vsegs to get better performance.  The error message alone is
>>> > > confusing
>>> > > > > so even if I am doing something wrong, the message should be
>>> > improved.
>>> > > > >
>>> > > > > My environment:
>>> > > > >
>>> > > > > AWS d2.8xlarge nodes
>>> > > > > - 24 2 TB disks
>>> > > > > - 252 GB RAM
>>> > > > > - 36 cores
>>> > > > > - Centos 6
>>> > > > > - 10 nodes
>>> > > > > - 1 admin node
>>> > > > > - 10GB network
>>> > > > > - 7 TB of data
>>> > > > > - Standard TPC-DS Queries
>>> > > > > - hawq_rm_memory_limit_perseg = 200gb
>>> > > > > - hawq_rm_stmt_vseg_memory = 16gb
>>> > > > > - Random distribution on all tables
>>> > > > >
>>> > > > > I'm tried reducing the statement memory but that doesn't change
>>> > > anything.
>>> > > > >
>>> > > > > Query 88 is a good example of this because it fails quickly.
>>> > > > >
>>> > > > > set hawq_rm_nvseg_perquery_perseg_limit=12;
>>> > > > >
>>> > > > > time psql -f 188.tpcds.88.sql
>>> > > > > SET
>>> > > > > Timing is on.
>>> > > > > SET
>>> > > > > Time: 0.157 ms
>>> > > > >  h8_30_to_9 | h9_to_9_30 | h9_30_to_10 | h10_to_10_30 |
>>> h10_30_to_11
>>> > |
>>> > > > > h11_to_11_30 | h11_30_to_12 | h12_to_12_30
>>> > > > > ------------+------------+-------------+--------------+-----
>>> > > > > ---------+--------------+--------------+--------------
>>> > > > >    16279055 |   32496701 |    32493080 |     48732586 |
>>>  48782652
>>> > |
>>> > > > > 28460584 |     28453299 |     32518016
>>> > > > > (1 row)
>>> > > > >
>>> > > > > Time: 259695.969 ms
>>> > > > >
>>> > > > > real 4m19.706s
>>> > > > > user 0m0.001s
>>> > > > > sys 0m0.003s
>>> > > > >
>>> > > > > Next:
>>> > > > > set hawq_rm_nvseg_perquery_perseg_limit=14;
>>> > > > >
>>> > > > > time psql -f 188.tpcds.88.sql
>>> > > > > SET
>>> > > > > Timing is on.
>>> > > > > SET
>>> > > > > Time: 0.171 ms
>>> > > > > psql:188.tpcds.88.sql:95: ERROR:  Error dispatching to seg25
>>> > > > > ip-172-21-13-189.ec2.internal:40000: connection pointer is NULL
>>> > > > > DETAIL:  Master unable to connect to seg25
>>> > > > > ip-172-21-13-189.ec2.internal:40000: FATAL:  unrecognized
>>> > > configuration
>>> > > > > parameter "ONETARY"
>>> > > > >
>>> > > > > real 0m8.787s
>>> > > > > user 0m0.003s
>>> > > > > sys 0m0.002s
>>> > > > >
>>> > > > >
>>> > > > > Jon Roberts
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

--001a113ed15c346843054425ad3a--