tajo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jihoon Son <ghoon...@gmail.com>
Subject Re: Big big query
Date Mon, 25 Aug 2014 09:13:13 GMT
Hi Christian,

thanks for your response and attatched log. However, it is still hard to
find the problem, because as you said, there are a couple of error messages
which are generated for different queries. In addition, your tajo looks to
be gone stale against master branch. So, even though I got some clues, but
cannot assure the reason of the problem.

So, it would be great if you share the compact log (which contains only the
log of the above query) after updating your tajo.

Thanks,
Jihoon


2014-08-25 15:52 GMT+09:00 Jinhang Choi <jinhang@linewalks.com>:

> Dear Christian,
>
> worker log indicates that "GC overhead limit exceeded."
>
> would you mind to extend worker's heap memory size at tajo-env.sh?
>
> (please refer to
> http://tajo.apache.org/docs/current/configuration/worker_configuration.html)
>
> Sincerely,
> ----------------------------------------------
> Jinhang Choi, CTO.
> Linewalks Inc. Seoul 137-860, Korea
> Office: +82 70 7702 3043
> FAX: +82 2 2055 0612
>
> -----Original Message-----
> *From:* "Christian Schwabe"<Christian.Schwabe@gmx.com>
> *To:* <user@tajo.apache.org>;
> *Cc:*
> *Sent:* 2014-08-25 (월) 15:33:15
> *Subject:* Re: Big big query
>
>
> Hello Hyunsik,
> sorry. My fault. I will send you another email with the attached logs.
>
> Best regards,
> Chris
>
> Am 25.08.2014 08:28:17, schrieb Hyunsik Choi:
>
> Hi Chris,
>
> As Jihoon mentioned, it would be better to find the problems if you attach
> master and worker logs.
>
> Thanks,
> hyunsik
>
>
> On Sun, Aug 24, 2014 at 4:17 PM, Christian Schwabe <
> Christian.Schwabe@gmx.com> wrote:
>
> Hello guys
>
> i started following query last night and this morning have seen that still
> ran the query with the fact that has the display of procent not changed and
> only ran on time. So I have to start again this morning reproduce the query
> to the error. The result you see in the table below:
> Running QueriesQueryIdQuery MasterStarted ProgressTimeStatussql Kill Query
> q_1408862421753_0001
> <http://christians-mbp.fritz.box:28080/querydetail.jsp?queryId=q_1408862421753_0001>
> christians-mbp.fritz.box2014-08-24 08:46:3345% 15 mins, 48 sec
> QUERY_RUNNINGINSERT INTO TMP_DFKKKO_DFKKOP select pos.validthru,
> pos.mandt, pos.opbel, pos.opupk, pos.opupz, pos.bukrs, pos.augrs,
> pos.abwkt, pos.hvorg, pos.tvorg, pos.kofiz, pos.hkont, pos.mwskz,
> pos.mwszkz, pos.xanza, pos.stakz, pos.astkz, pos.opsta, pos.infoz,
> pos.inkps, pos.betrh, pos.studt, ko.fikey, ko.blart, ko.herkf, ko.stbel,
> ko.storb, ko.ernam, ko.cpudt, ko.cputm, ko.bldat, ko.budat from dfkkop_hist
> pos left join dfkkopw_hist wdh on ( pos.validthru = wdh.validthru and
> pos.mandt = wdh.mandt and pos.opbel = wdh.opbel and pos.whgrp = wdh.whgrp )
> inner join dfkkko_hist ko on ( pos.validthru = ko.validthru and pos.mandt =
> ko.mandt and pos.opbel = ko.opbel )
>
> Second Screenshot:
> Running QueriesQueryIdQuery MasterStarted ProgressTimeStatussql Kill Query
> q_1408862421753_0001
> <http://christians-mbp.fritz.box:28080/querydetail.jsp?queryId=q_1408862421753_0001>
> christians-mbp.fritz.box2014-08-24 08:46:3343% 23 mins, 21 sec
> QUERY_RUNNINGINSERT INTO TMP_DFKKKO_DFKKOP select pos.validthru,
> pos.mandt, pos.opbel, pos.opupk, pos.opupz, pos.bukrs, pos.augrs,
> pos.abwkt, pos.hvorg, pos.tvorg, pos.kofiz, pos.hkont, pos.mwskz,
> pos.mwszkz, pos.xanza, pos.stakz, pos.astkz, pos.opsta, pos.infoz,
> pos.inkps, pos.betrh, pos.studt, ko.fikey, ko.blart, ko.herkf, ko.stbel,
> ko.storb, ko.ernam, ko.cpudt, ko.cputm, ko.bldat, ko.budat from dfkkop_hist
> pos left join dfkkopw_hist wdh on ( pos.validthru = wdh.validthru and
> pos.mandt = wdh.mandt and pos.opbel = wdh.opbel and pos.whgrp = wdh.whgrp )
> inner join dfkkko_hist ko on ( pos.validthru = ko.validthru and pos.mandt =
> ko.mandt and pos.opbel = ko.opbel )
>
> Third Screenshot:
>
> Finished QueriesQueryIdQuery MasterStarted FinishedTimeStatussql
> q_1408862421753_0001
> <http://christians-mbp.fritz.box:28080/querydetail.jsp?queryId=q_1408862421753_0001>
> christians-mbp.fritz.box2014-08-24 08:46:33-- QUERY_RUNNINGINSERT INTO
> TMP_DFKKKO_DFKKOP select pos.validthru, pos.mandt, pos.opbel, pos.opupk,
> pos.opupz, pos.bukrs, pos.augrs, pos.abwkt, pos.hvorg, pos.tvorg,
> pos.kofiz, pos.hkont, pos.mwskz, pos.mwszkz, pos.xanza, pos.stakz,
> pos.astkz, pos.opsta, pos.infoz, pos.inkps, pos.betrh, pos.studt, ko.fikey,
> ko.blart, ko.herkf, ko.stbel, ko.storb, ko.ernam, ko.cpudt, ko.cputm,
> ko.bldat, ko.budat from dfkkop_hist pos left join dfkkopw_hist wdh on (
> pos.validthru = wdh.validthru and pos.mandt = wdh.mandt and pos.opbel =
> wdh.opbel and pos.whgrp = wdh.whgrp ) inner join dfkkko_hist ko on (
> pos.validthru = ko.validthru and pos.mandt = ko.mandt and pos.opbel =
> ko.opbel )
>
> As you can see, the query is still running but there is no image data and
> progress more.
>
> I attached a log in which you can see the output in the console. Striking
> here is the display of the percentage jumps sometimes and not further
> altered towards the end and remains constant.
> The data size of the tables to which I JOINS by lead is for dfkkop_hist
> 5,83GB, dfkkopw_hist 2,47GB and dfkkko_hist 2.35 GB. As I write this, the
> description of the query is still running.
> I know these are large amounts of data, but I would have expected which
> thus constitutes the colloquial no problem. Can you imagine why it comes
> here to this problem?
>
>
>
>
> Best regards,
> Chris
>
>
>
>
>



-- 
Jihoon Son

Database & Information Systems Group,
Prof. Yon Dohn Chung Lab.
Dept. of Computer Science & Engineering,
Korea University
1, 5-ga, Anam-dong, Seongbuk-gu,
Seoul, 136-713, Republic of Korea

Tel : +82-2-3290-3580
E-mail : jihoonson@korea.ac.kr

Mime
View raw message