Mailing-List: contact dev-help@hawq.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hawq.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <CAH_aXGNbn_p8QK-kjLH378pcijezkZAG9F_V7XE8NGY8C9hFnw@mail.gmail.com>
References: <CAH_aXGMyWw=ZVUx2opZA+wfgMk6mf6WBhqarEHv8ObT=3RZevA@mail.gmail.com>
 <CABQrizc8SBfcnvYNP3+5uxaxbRSyG8mB_ZvnXZ_0k6YL_tOVhg@mail.gmail.com>
 <CAMjXoANyNw406aiD0yDO8OaN36mgQO=2E3uPLffw6v-FvsHDdw@mail.gmail.com>
 <CAH_aXGMaNhTOHUJ-zTjnt-jVKjba=RBBmtdbtXj6it6g+TC6nA@mail.gmail.com>
 <CAMjXoAObWvLjjHhnXSMzHyytC06Hz+tNNQON7gBiYbyEuxw=sA@mail.gmail.com>
 <CA+F1ufkCVBpwCiNu7u0JKiDxreRd-0cWaQkUD2vkCTBeFK6qOg@mail.gmail.com>
 <CAH=zMbpX=Xek8id2XgER=TYYiJH6oLibpTk7z+zR4+u7nBw=_A@mail.gmail.com>
 <CA+F1uf=rXjgNuk=XJA+=wfypT48_nMuXuDja+TKuuqniqhMx4g@mail.gmail.com> <CAH_aXGNbn_p8QK-kjLH378pcijezkZAG9F_V7XE8NGY8C9hFnw@mail.gmail.com>
From: Yi Jin <yjin@pivotal.io>
Date: Thu, 29 Dec 2016 10:08:53 +1100
Message-ID: <CAMjXoAMgXBmF2Q29Hb=tz4qSScCpAk3++GXt96ANDNNDh8mh+A@mail.gmail.com>
Subject: Re: unrecognized configuration parameter "ONETARY"
To: dev <dev@hawq.incubator.apache.org>
Cc: Radar Da lei <rlei@pivotal.io>
Content-Type: multipart/alternative; boundary=001a114a73aa74f8d90544c00d86
archived-at: Wed, 28 Dec 2016 23:09:08 -0000

--001a114a73aa74f8d90544c00d86
Content-Type: text/plain; charset=UTF-8

Jon,

Thank you, yes, I am looking into this bug now.

Best,
Yi

On Thu, Dec 29, 2016 at 2:33 AM, Jon Roberts <jroberts@pivotal.io> wrote:

> I created a Jira for this bug:
> https://issues.apache.org/jira/browse/HAWQ-1244
>
> Jon Roberts
> Principal Engineer | jroberts@pivotal.io | 615-426-8661
>
> On Wed, Dec 21, 2016 at 12:58 AM, Ming Li <mli@pivotal.io> wrote:
>
> > Thanks Radar,
> >
> > It is strange that 'unrecognized configuration' will issue FATAL error, I
> > config in hawq-site.xml, and grep pg_log on master and segment date
> > directory, no FATAL error of 'unrecognized configuration'. Any ideas?
> >
> > On Wed, Dec 21, 2016 at 2:45 PM, Radar Da lei <rlei@pivotal.io> wrote:
> >
> > > Hi Ming,
> > >
> > > 'hawq_lc_monetary' is only used to store user defined '--lc-monetary'
> > value
> > > then pass below option to initdb:
> > >       '--lc-monetary=${hawq_lc_monetary} '
> > >
> > > So finally we give '--lc-monetary=en_US.utf8 ' to initdb. Base on this
> I
> > > don't think the GUC 'ONETARY' have any dependencies with hawq admin
> > > tools. Thanks.
> > >
> > > Regards,
> > > Radar
> > >
> > > On Tue, Dec 20, 2016 at 10:18 AM, Ming Li <mli@pivotal.io> wrote:
> > >
> > >> Hi Radar,
> > >>
> > >> It seems that the config param 'hawq_lc_monetary' can only be
> recognized
> > >> by hawq admin tools in python, and set hawq guc 'lc-monetary' finally.
> > >>
> > >> My question is: Why not directly using 'lc-monetary' instead of '
> > >> hawq_lc_monetary' in the hawq admin tools? Is there any possible to
> > >> conflict?
> > >>
> > >> Thanks.
> > >>
> > >> On Tue, Dec 20, 2016 at 6:45 AM, Yi Jin <yjin@pivotal.io> wrote:
> > >>
> > >>> Thank you jon, I will specially recheck the guc population logic. Yi
> > >>>
> > >>> On Tue, Dec 20, 2016 at 4:07 AM, Jon Roberts <jroberts@pivotal.io>
> > >>> wrote:
> > >>>
> > >>> > The segment actually has this error:
> > >>> > 2016-12-14 13:47:34.760839
> > >>> > UTC,"gpadmin","gpadmin",p737499,th542214432,"172.21.
> > >>> > 13.196","40327",2016-12-14
> > >>> > 13:47:34 UTC,0,con23798,,seg-10000,,,,,
> "FATAL","42704","unrecognized
> > >>> > configuration parameter ""ONETARY""",,,,,,,0,,"guc.c",10006,
> > >>> >
> > >>> > This made me check out the configs because the error was raised
> from
> > >>> > guc.c.  I found that /usr/local/hawq/etc/_mgmt_config has
> "onetary"
> > >>> text:
> > >>> > hawq_lc_monetary=en_US.utf8
> > >>> >
> > >>> > I'm using nodes with 24 drives so I have 24 temp directories set
> for
> > >>> both
> > >>> > the master and the segments.  Thinking that possibly the problem is
> > >>> related
> > >>> > to the number of vSegs * the size of the config file exceeding a
> > >>> variable
> > >>> > size, I decided to reduce the temp directories down to just two for
> > >>> both
> > >>> > the master and segments.
> > >>> >
> > >>> > After restarting HAWQ, I could get the query to use 16 and even 24
> > >>> vSegs
> > >>> > without a problem.
> > >>> >
> > >>> > So maybe the guc logic needs revisiting to make sure there isn't an
> > >>> issue
> > >>> > with parsing the GUCs when there are many vSegs and many temp
> > >>> directories.
> > >>> > I would think this would be a problem even with the default number
> of
> > >>> vSegs
> > >>> > per host on really large clusters.
> > >>> >
> > >>> >
> > >>> > Jon Roberts
> > >>> > Principal Engineer | jroberts@pivotal.io | 615-426-8661
> > >>> <(615)%20426-8661>
> > >>> >
> > >>> > On Mon, Dec 19, 2016 at 4:44 AM, Yi Jin <yjin@pivotal.io> wrote:
> > >>> >
> > >>> > > Hi Jon,
> > >>> > >
> > >>> > > I think there is no ONTARY configuration item, so maybe I need
> the
> > >>> full
> > >>> > log
> > >>> > > containing that error to check the error routine.
> > >>> > >
> > >>> > > Best,
> > >>> > > Yi
> > >>> > >
> > >>> > > On Mon, Dec 19, 2016 at 5:39 PM, Paul Guo <paulguo@gmail.com>
> > wrote:
> > >>> > >
> > >>> > > > You could grep the log to see whether there is "ONETARY"
> setting
> > >>> before
> > >>> > > > this error occurs, and also grep configuration files and
> related
> > >>> test
> > >>> > > files
> > >>> > > > to find who tried to set this.
> > >>> > > >
> > >>> > > > 2016-12-14 22:36 GMT+08:00 Jon Roberts <jroberts@pivotal.io>:
> > >>> > > >
> > >>> > > > > I'm getting the error message: unrecognized configuration
> > >>> parameter
> > >>> > > > > "ONETARY" with a few of the TPC-DS queries where I'm
> increasing
> > >>> the
> > >>> > > > number
> > >>> > > > > of vsegs to get better performance.  The error message alone
> is
> > >>> > > confusing
> > >>> > > > > so even if I am doing something wrong, the message should be
> > >>> > improved.
> > >>> > > > >
> > >>> > > > > My environment:
> > >>> > > > >
> > >>> > > > > AWS d2.8xlarge nodes
> > >>> > > > > - 24 2 TB disks
> > >>> > > > > - 252 GB RAM
> > >>> > > > > - 36 cores
> > >>> > > > > - Centos 6
> > >>> > > > > - 10 nodes
> > >>> > > > > - 1 admin node
> > >>> > > > > - 10GB network
> > >>> > > > > - 7 TB of data
> > >>> > > > > - Standard TPC-DS Queries
> > >>> > > > > - hawq_rm_memory_limit_perseg = 200gb
> > >>> > > > > - hawq_rm_stmt_vseg_memory = 16gb
> > >>> > > > > - Random distribution on all tables
> > >>> > > > >
> > >>> > > > > I'm tried reducing the statement memory but that doesn't
> change
> > >>> > > anything.
> > >>> > > > >
> > >>> > > > > Query 88 is a good example of this because it fails quickly.
> > >>> > > > >
> > >>> > > > > set hawq_rm_nvseg_perquery_perseg_limit=12;
> > >>> > > > >
> > >>> > > > > time psql -f 188.tpcds.88.sql
> > >>> > > > > SET
> > >>> > > > > Timing is on.
> > >>> > > > > SET
> > >>> > > > > Time: 0.157 ms
> > >>> > > > >  h8_30_to_9 | h9_to_9_30 | h9_30_to_10 | h10_to_10_30 |
> > >>> h10_30_to_11
> > >>> > |
> > >>> > > > > h11_to_11_30 | h11_30_to_12 | h12_to_12_30
> > >>> > > > > ------------+------------+-------------+--------------+-----
> > >>> > > > > ---------+--------------+--------------+--------------
> > >>> > > > >    16279055 |   32496701 |    32493080 |     48732586 |
> > >>>  48782652
> > >>> > |
> > >>> > > > > 28460584 |     28453299 |     32518016
> > >>> > > > > (1 row)
> > >>> > > > >
> > >>> > > > > Time: 259695.969 ms
> > >>> > > > >
> > >>> > > > > real 4m19.706s
> > >>> > > > > user 0m0.001s
> > >>> > > > > sys 0m0.003s
> > >>> > > > >
> > >>> > > > > Next:
> > >>> > > > > set hawq_rm_nvseg_perquery_perseg_limit=14;
> > >>> > > > >
> > >>> > > > > time psql -f 188.tpcds.88.sql
> > >>> > > > > SET
> > >>> > > > > Timing is on.
> > >>> > > > > SET
> > >>> > > > > Time: 0.171 ms
> > >>> > > > > psql:188.tpcds.88.sql:95: ERROR:  Error dispatching to seg25
> > >>> > > > > ip-172-21-13-189.ec2.internal:40000: connection pointer is
> > NULL
> > >>> > > > > DETAIL:  Master unable to connect to seg25
> > >>> > > > > ip-172-21-13-189.ec2.internal:40000: FATAL:  unrecognized
> > >>> > > configuration
> > >>> > > > > parameter "ONETARY"
> > >>> > > > >
> > >>> > > > > real 0m8.787s
> > >>> > > > > user 0m0.003s
> > >>> > > > > sys 0m0.002s
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > Jon Roberts
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

--001a114a73aa74f8d90544c00d86--