drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andries Engelbrecht (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3721) Regarding drill with big file
Date Fri, 28 Aug 2015 14:51:46 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14719296#comment-14719296
] 

Andries Engelbrecht commented on DRILL-3721:
--------------------------------------------

See what the query memory per node is set at and increase it to see if it resolves your problem.

The parameter is  planner.memory.max_query_memory_per_node

Query sys.options to see what it is set as and use alter system to modify.

https://drill.apache.org/docs/configuring-drill-memory/

https://drill.apache.org/docs/alter-system/

https://drill.apache.org/docs/configuration-options-introduction/


> Regarding drill with big file
> -----------------------------
>
>                 Key: DRILL-3721
>                 URL: https://issues.apache.org/jira/browse/DRILL-3721
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: kunal
>         Attachments: sample.json, sqlline.log
>
>
> I am new to apache drill. I have configured apache drill on machine with centos.
> "DRILL_MAX_DIRECT_MEMORY" = 25g
> "DRILL_HEAP" = 4g
> I have a 600 mb and 3 gb json file [sample file attached]. When i fire query on relativly
small size file everything works fine but as I fire same query with 600 mb and 3 gb files
it gives following error[stack trace attached].
> Query - 
> select tbl5.product_id product_id,tbl5.gender gender,tbl5.item_number item_number,tbl5.price
price,tbl5.description description,tbl5.color_swatch.image image,tbl5.color_swatch.color color
from
> (select tbl4.product_id product_id,tbl4.gender gender,tbl4.item_number item_number,tbl4.price
price,tbl4.size.description description,FLATTEN(tbl4.size.color_swatch) color_swatch from
> (select tbl3.product_id product_id,tbl3.catalog_item.gender gender,tbl3.catalog_item.item_number
item_number,tbl3.catalog_item.price price,FLATTEN(tbl3.catalog_item.size) size from 
> (select tbl2.product.product_id as product_id,FLATTEN(tbl2.product.catalog_item) as catalog_item
from 
> (select FLATTEN(tbl1.catalog.product) product from dfs.root.`demo.json` tbl1) tbl2) tbl3)
tbl4) tbl5
> --------------------------------------------------------------------------------------------------
> Error -
> SYSTEM ERROR: IllegalArgumentException: initialCapacity: -2147483648 (expectd: 0+)
> Fragment 0:0
> [Error Id: 60cf1b95-762d-4a0d-8cae-a2db418d4ea9 on sinhagad:31010]
> --------------------------------------------------------------------------------------------------
> 1) Am i doing someting wrong or missing something ( probably because i am not using cluster
?? ).
> Please guide me through this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message