drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Victoria Markman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-2293) CTAS does not clean up when it fails
Date Tue, 18 Aug 2015 17:49:45 GMT

    [ https://issues.apache.org/jira/browse/DRILL-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701685#comment-14701685
] 

Victoria Markman commented on DRILL-2293:
-----------------------------------------

One more example, CTAS fails with out of memory:
{code}
0: jdbc:drill:schema=dfs> create table lineitem as select
. . . . . . . . . . . . >     cast(columns[0] as int) l_orderkey,
. . . . . . . . . . . . >     cast(columns[1] as int) l_partkey,
. . . . . . . . . . . . >     cast(columns[2] as int) l_suppkey,
. . . . . . . . . . . . >     cast(columns[3] as int) l_linenumber,
. . . . . . . . . . . . >     cast(columns[4] as double) l_quantity,
. . . . . . . . . . . . >     cast(columns[5] as double) l_extendedprice,
. . . . . . . . . . . . >     cast(columns[6] as double) l_discount,
. . . . . . . . . . . . >     cast(columns[7] as double) l_tax,
. . . . . . . . . . . . >     cast(columns[8] as varchar(200)) l_returnflag,
. . . . . . . . . . . . >     cast(columns[9] as varchar(200)) l_linestatus,
. . . . . . . . . . . . >     cast(columns[10] as date) l_shipdate,
. . . . . . . . . . . . >     cast(columns[11] as date) l_commitdate,
. . . . . . . . . . . . >     cast(columns[12] as date) l_receiptdate,
. . . . . . . . . . . . >     cast(columns[13] as varchar(200)) l_shipinstruct,
. . . . . . . . . . . . >     cast(columns[14] as varchar(200)) l_shipmode,
. . . . . . . . . . . . >     cast(columns[15] as varchar(200)) l_comment
. . . . . . . . . . . . > from `lineitem.dat`;
Error: RESOURCE ERROR: One or more nodes ran out of memory while executing the query.
Fragment 1:10
[Error Id: 11084315-5388-4500-b165-642a5f595ebf on atsqa4-133.qa.lab:31010] (state=,code=0)
{code}

Table gets created and an attempt to query table results in an error:
{code}
0: jdbc:drill:schema=dfs> select count(*) from lineitem;
Error: SYSTEM ERROR: IllegalArgumentException: maprfs:///drill/testdata/tpch10/lineitem/1_8_0.parquet
is not a Parquet file (too small)
[Error Id: 4836e16c-1666-4783-a130-b298fc0e464c on atsqa4-133.qa.lab:31010] (state=,code=0)
{code}

> CTAS does not clean up when it fails
> ------------------------------------
>
>                 Key: DRILL-2293
>                 URL: https://issues.apache.org/jira/browse/DRILL-2293
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Parquet
>            Reporter: Rahul Challapalli
>            Assignee: Steven Phillips
>             Fix For: 1.3.0
>
>
> git.commit.id.abbrev=6676f2d
> Data Set :
> {code}
> {
>   "id" : 1,
>   "map":{"rm": [
>     {"mapid":"m1","mapvalue":{"col1":1,"col2":[0,1,2,3,4,5]},"rptd": [{ "a": "foo"},{"b":"boo"}]},
>     {"mapid":"m2","mapvalue":{"col1":0,"col2":[]},"rptd": [{ "a": "bar"},{"c":1},{"d":4.5}]}
>   ]}
> }
> {code}
> The below query fails :
> {code}
> create table rep_map as select d.map from `temp.json` d;
> Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75
on qa-node190.qa.lab:31010 ]
> {code}
> However drill created a folder 'rep_map' and the folder contained a broken parquet file.

> {code}
> create table rep_map as select d.map from `temp.json` d;
> +------------+------------+
> |     ok     |  summary   |
> +------------+------------+
> | false      | Table 'rep_map' already exists. |
> {code}
> Drill should clean up properly in case of a failure.
> I raised a different issue for the actual failure.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message