drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khurram Faraaz (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3468) CTAS IOB
Date Tue, 07 Jul 2015 21:25:04 GMT
Khurram Faraaz created DRILL-3468:
-------------------------------------

             Summary: CTAS IOB
                 Key: DRILL-3468
                 URL: https://issues.apache.org/jira/browse/DRILL-3468
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 1.1.0
            Reporter: Khurram Faraaz
            Assignee: Chris Westin
            Priority: Critical


I am seeing a IOB when I use same table name in CTAS, after deleting the previously create
parquet file.

{code}
0: jdbc:drill:schema=dfs.tmp> CREATE TABLE tbl_allData AS SELECT CAST(columns[0] as INT
), CAST(columns[1] as BIGINT ), CAST(columns[2] as CHAR(2) ), CAST(columns[3] as VARCHAR(52)
), CAST(columns[4] as TIMESTAMP ), CAST(columns[5] as DATE ), CAST(columns[6] as BOOLEAN ),
CAST(columns[7] as DOUBLE), CAST( columns[8] as TIME) FROM `allData.csv`;
+-----------+----------------------------+
| Fragment  | Number of records written  |
+-----------+----------------------------+
| 0_0       | 11196                      |
+-----------+----------------------------+
1 row selected (1.864 seconds)
{code}

Remove the parquet file that was created by the above CTAS.

{code}
[root@centos-01 aggregates]# hadoop fs -ls /tmp/tbl_allData
Found 1 items
-rwxr-xr-x   3 mapr mapr     397868 2015-07-07 21:08 /tmp/tbl_allData/0_0_0.parquet
[root@centos-01 aggregates]# hadoop fs -rm /tmp/tbl_allData/0_0_0.parquet
15/07/07 21:10:47 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead,
use dfs.bytes-per-checksum
15/07/07 21:10:47 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval
= 0 minutes, Emptier interval = 0 minutes.
Deleted /tmp/tbl_allData/0_0_0.parquet
{code}

I see a IOB when I CTAS with same table name as the one that was removed in the above step.

{code}
0: jdbc:drill:schema=dfs.tmp> CREATE TABLE tbl_allData AS SELECT CAST(columns[0] as INT
), CAST(columns[1] as BIGINT ), CAST(columns[2] as CHAR(2) ), CAST(columns[3] as VARCHAR(52)
), CAST(columns[4] as TIMESTAMP ), CAST(columns[5] as DATE ), CAST(columns[6] as BOOLEAN ),
CAST(columns[7] as DOUBLE), CAST( columns[8] as TIME) FROM `lessData.csv`;
Error: SYSTEM ERROR: IndexOutOfBoundsException: Index: 0, Size: 0


[Error Id: 6d6df8e9-699c-4475-8ad3-183c0a91dc99 on centos-02.qa.lab:31010] (state=,code=0)
{code}

stack trace from drillbit.log

{code}
org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment
initialization: Failure while trying to check if a table or view with given name [tbl_allData]
already exists in schema [dfs.tmp]: Index: 0, Size: 0
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:253) [drill-java-exec-1.1.0.jar:1.1.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Failure while trying
to check if a table or view with given name [tbl_allData] already exists in schema [dfs.tmp]:
Index: 0, Size: 0
        at org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:222)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.planner.sql.handlers.CreateTableHandler.getPlan(CreateTableHandler.java:88)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:903) [drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:242) [drill-java-exec-1.1.0.jar:1.1.0]
        ... 3 common frames omitted
Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
        at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_45]
        at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_45]
        at org.apache.drill.exec.store.dfs.FileSelection.getFirstPath(FileSelection.java:100)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.store.dfs.BasicFormatMatcher.isReadable(BasicFormatMatcher.java:75)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:303)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:118)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:241)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:219)
~[drill-java-exec-1.1.0.jar:1.1.0]
        ... 7 common frames omitted
2015-07-07 21:12:14,643 [2a63bf51-09ff-467b-1aab-df2b5030e9b9:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- foreman cleaning up.

2015-07-07 21:12:14,648 [2a63bf51-09ff-467b-1aab-df2b5030e9b9:foreman] ERROR o.a.drill.exec.work.foreman.Foreman
- SYSTEM ERROR: IndexOutOfBoundsException: Index: 0, Size: 0


[Error Id: 6d6df8e9-699c-4475-8ad3-183c0a91dc99 on centos-02.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IndexOutOfBoundsException:
Index: 0, Size: 0


[Error Id: 6d6df8e9-699c-4475-8ad3-183c0a91dc99 on centos-02.qa.lab:31010]
        at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523)
~[drill-common-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:737)
[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:839)
[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:781)
[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.common.EventProcessor.sendEvent(EventProcessor.java:73) [drill-common-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:783)
[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:892) [drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:253) [drill-java-exec-1.1.0.jar:1.1.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during
fragment initialization: Failure while trying to check if a table or view with given name
[tbl_allData] already exists in schema [dfs.tmp]: Index: 0, Size: 0
        ... 4 common frames omitted
Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Failure while trying
to check if a table or view with given name [tbl_allData] already exists in schema [dfs.tmp]:
Index: 0, Size: 0
        at org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:222)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.planner.sql.handlers.CreateTableHandler.getPlan(CreateTableHandler.java:88)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:903) [drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:242) [drill-java-exec-1.1.0.jar:1.1.0]
        ... 3 common frames omitted
Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
        at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_45]
        at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_45]
        at org.apache.drill.exec.store.dfs.FileSelection.getFirstPath(FileSelection.java:100)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.store.dfs.BasicFormatMatcher.isReadable(BasicFormatMatcher.java:75)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:303)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:118)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:241)
~[drill-java-exec-1.1.0.jar:1.1.0]
        at org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:219)
~[drill-java-exec-1.1.0.jar:1.1.0]
        ... 7 common frames omitted
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message