drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Muhammad Gelbana (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5300) SYSTEM ERROR: IllegalStateException: Memory was leaked by query while querying parquet files
Date Sun, 14 May 2017 10:56:04 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009677#comment-16009677
] 

Muhammad Gelbana commented on DRILL-5300:
-----------------------------------------

[~kkhatua], I tried it once v1.10 was released but the issue wasn't solved. I still had to
clone this [repo|https://github.com/dain/snappy.git], build it, and include the result JAR
with Drill in the jard/3rdparty folder. Forgive me for the late reply.

> SYSTEM ERROR: IllegalStateException: Memory was leaked by query while querying parquet
files
> --------------------------------------------------------------------------------------------
>
>                 Key: DRILL-5300
>                 URL: https://issues.apache.org/jira/browse/DRILL-5300
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.9.0
>         Environment: OS: Linux
>            Reporter: Muhammad Gelbana
>         Attachments: both_queries_logs.zip
>
>
> Running the following query against parquet files (I modified some values for privacy
reasons)
> {code:title=Query causing the long logs|borderStyle=solid}
> SELECT AL4.NAME, AL5.SEGMENT2, SUM(AL1.AMOUNT), AL2.ATTRIBUTE4, AL2.XXXXXXX_XXXXXXXX_CODE,
AL8.D_BU, AL8.F_PL, AL18.COUNTRY, AL13.COUNTRY, AL11.NAME FROM dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XX/RA_XXXX_TRX_LINE_GL_DIST_ALL`
AL1, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XX/RA_XXXXOMER_TRX_ALL` AL2,
dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_FIN_COMMON/GL_XXXXXXX` AL3, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_HR_COMMON/HR_ALL_ORGANIZATION_UNITS`
AL4, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_FIN_COMMON/GL_CODE_COMBINATIONS`
AL5, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXXXXXXX/XXAT_AR_MU_TAB` AL8, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_FIN_COMMON/GL_XXXXXXX`
AL11, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXXX_XXXXS`
AL12, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_LOCATIONS`
AL13, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXX_XXXX_XXXXS_ALL`
AL14, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXX_XXXX_USES_ALL`
AL15, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXX_XXXX_XXXXS_ALL`
AL16, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXX_XXXX_USES_ALL`
AL17, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_LOCATIONS`
AL18, dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXXX_XXXXS`
AL19 WHERE (AL2.SHIP_TO_XXXX_USE_ID = AL15.XXXX_USE_ID AND AL15.XXXX_XXXX_XXXX_ID = AL14.XXXX_XXXX_XXXX_ID
AND AL14.XXXXX_XXXX_ID = AL12.XXXXX_XXXX_ID AND AL12.LOCATION_ID = AL13.LOCATION_ID AND AL17.XXXX_XXXX_XXXX_ID
= AL16.XXXX_XXXX_XXXX_ID AND AL16.XXXXX_XXXX_ID = AL19.XXXXX_XXXX_ID AND AL19.LOCATION_ID
= AL18.LOCATION_ID AND AL2.BILL_TO_XXXX_USE_ID = AL17.XXXX_USE_ID AND AL2.SET_OF_XXXXX_ID
= AL3.SET_OF_XXXXX_ID AND AL1.CODE_COMBINATION_ID = AL5.CODE_COMBINATION_ID AND AL5.SEGMENT4
= AL8.MU AND AL1.SET_OF_XXXXX_ID = AL11.SET_OF_XXXXX_ID AND AL2.ORG_ID = AL4.ORGANIZATION_ID
AND AL2.XXXXOMER_TRX_ID = AL1.XXXXOMER_TRX_ID) AND ((AL5.SEGMENT2 = '400001' AND AL1.AMOUNT
<> 0 AND AL4.NAME IN ('XXX-XX-XXXX', 'XXX-XX-XXXX', 'XXX-XX-XXXX', 'XXX-XX-XXXX', 'XXX-XX-XXXX',
'XXX-XX-XXXX', 'XXX-XX-XXXX', 'XXX-XX-XXXX', 'XXX-XX-XXXX') AND AL3.NAME like '%-PR-%')) GROUP
BY AL4.NAME, AL5.SEGMENT2, AL2.ATTRIBUTE4, AL2.XXXXXXX_XXXXXXXX_CODE, AL8.D_BU, AL8.F_PL,
AL18.COUNTRY, AL13.COUNTRY, AL11.NAME
> {code}
> {code:title=Query causing the short logs|borderStyle=solid}
> SELECT AL11.NAME
> FROM
> dfs.`/XXXXXXX/XXXXXXX/XXXXXXX/data/../parquet/XXX_XXX_COMMON/GL_XXXXXXX` XXXX
> LIMIT 10
> {code}
> This issue may be a duplicate for [this one|https://issues.apache.org/jira/browse/DRILL-4398]
but I created a new one based on [this suggestion|https://issues.apache.org/jira/browse/DRILL-4398?focusedCommentId=15884846&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15884846].



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message