Return-Path: X-Original-To: apmail-drill-issues-archive@minotaur.apache.org Delivered-To: apmail-drill-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B6FCA18171 for ; Tue, 7 Jul 2015 21:25:04 +0000 (UTC) Received: (qmail 57338 invoked by uid 500); 7 Jul 2015 21:25:04 -0000 Delivered-To: apmail-drill-issues-archive@drill.apache.org Received: (qmail 57309 invoked by uid 500); 7 Jul 2015 21:25:04 -0000 Mailing-List: contact issues-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list issues@drill.apache.org Received: (qmail 57299 invoked by uid 99); 7 Jul 2015 21:25:04 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Jul 2015 21:25:04 +0000 Date: Tue, 7 Jul 2015 21:25:04 +0000 (UTC) From: "Khurram Faraaz (JIRA)" To: issues@drill.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (DRILL-3468) CTAS IOB MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Khurram Faraaz created DRILL-3468: ------------------------------------- Summary: CTAS IOB Key: DRILL-3468 URL: https://issues.apache.org/jira/browse/DRILL-3468 Project: Apache Drill Issue Type: Bug Components: Execution - Flow Affects Versions: 1.1.0 Reporter: Khurram Faraaz Assignee: Chris Westin Priority: Critical I am seeing a IOB when I use same table name in CTAS, after deleting the previously create parquet file. {code} 0: jdbc:drill:schema=dfs.tmp> CREATE TABLE tbl_allData AS SELECT CAST(columns[0] as INT ), CAST(columns[1] as BIGINT ), CAST(columns[2] as CHAR(2) ), CAST(columns[3] as VARCHAR(52) ), CAST(columns[4] as TIMESTAMP ), CAST(columns[5] as DATE ), CAST(columns[6] as BOOLEAN ), CAST(columns[7] as DOUBLE), CAST( columns[8] as TIME) FROM `allData.csv`; +-----------+----------------------------+ | Fragment | Number of records written | +-----------+----------------------------+ | 0_0 | 11196 | +-----------+----------------------------+ 1 row selected (1.864 seconds) {code} Remove the parquet file that was created by the above CTAS. {code} [root@centos-01 aggregates]# hadoop fs -ls /tmp/tbl_allData Found 1 items -rwxr-xr-x 3 mapr mapr 397868 2015-07-07 21:08 /tmp/tbl_allData/0_0_0.parquet [root@centos-01 aggregates]# hadoop fs -rm /tmp/tbl_allData/0_0_0.parquet 15/07/07 21:10:47 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum 15/07/07 21:10:47 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes. Deleted /tmp/tbl_allData/0_0_0.parquet {code} I see a IOB when I CTAS with same table name as the one that was removed in the above step. {code} 0: jdbc:drill:schema=dfs.tmp> CREATE TABLE tbl_allData AS SELECT CAST(columns[0] as INT ), CAST(columns[1] as BIGINT ), CAST(columns[2] as CHAR(2) ), CAST(columns[3] as VARCHAR(52) ), CAST(columns[4] as TIMESTAMP ), CAST(columns[5] as DATE ), CAST(columns[6] as BOOLEAN ), CAST(columns[7] as DOUBLE), CAST( columns[8] as TIME) FROM `lessData.csv`; Error: SYSTEM ERROR: IndexOutOfBoundsException: Index: 0, Size: 0 [Error Id: 6d6df8e9-699c-4475-8ad3-183c0a91dc99 on centos-02.qa.lab:31010] (state=,code=0) {code} stack trace from drillbit.log {code} org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Failure while trying to check if a table or view with given name [tbl_allData] already exists in schema [dfs.tmp]: Index: 0, Size: 0 at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:253) [drill-java-exec-1.1.0.jar:1.1.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Failure while trying to check if a table or view with given name [tbl_allData] already exists in schema [dfs.tmp]: Index: 0, Size: 0 at org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:222) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.planner.sql.handlers.CreateTableHandler.getPlan(CreateTableHandler.java:88) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:903) [drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:242) [drill-java-exec-1.1.0.jar:1.1.0] ... 3 common frames omitted Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_45] at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_45] at org.apache.drill.exec.store.dfs.FileSelection.getFirstPath(FileSelection.java:100) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.store.dfs.BasicFormatMatcher.isReadable(BasicFormatMatcher.java:75) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:303) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:118) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:241) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:219) ~[drill-java-exec-1.1.0.jar:1.1.0] ... 7 common frames omitted 2015-07-07 21:12:14,643 [2a63bf51-09ff-467b-1aab-df2b5030e9b9:foreman] INFO o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. 2015-07-07 21:12:14,648 [2a63bf51-09ff-467b-1aab-df2b5030e9b9:foreman] ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: IndexOutOfBoundsException: Index: 0, Size: 0 [Error Id: 6d6df8e9-699c-4475-8ad3-183c0a91dc99 on centos-02.qa.lab:31010] org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IndexOutOfBoundsException: Index: 0, Size: 0 [Error Id: 6d6df8e9-699c-4475-8ad3-183c0a91dc99 on centos-02.qa.lab:31010] at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523) ~[drill-common-1.1.0.jar:1.1.0] at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:737) [drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:839) [drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:781) [drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.common.EventProcessor.sendEvent(EventProcessor.java:73) [drill-common-1.1.0.jar:1.1.0] at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:783) [drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:892) [drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:253) [drill-java-exec-1.1.0.jar:1.1.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Failure while trying to check if a table or view with given name [tbl_allData] already exists in schema [dfs.tmp]: Index: 0, Size: 0 ... 4 common frames omitted Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Failure while trying to check if a table or view with given name [tbl_allData] already exists in schema [dfs.tmp]: Index: 0, Size: 0 at org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:222) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.planner.sql.handlers.CreateTableHandler.getPlan(CreateTableHandler.java:88) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:903) [drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:242) [drill-java-exec-1.1.0.jar:1.1.0] ... 3 common frames omitted Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_45] at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_45] at org.apache.drill.exec.store.dfs.FileSelection.getFirstPath(FileSelection.java:100) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.store.dfs.BasicFormatMatcher.isReadable(BasicFormatMatcher.java:75) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:303) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:118) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:241) ~[drill-java-exec-1.1.0.jar:1.1.0] at org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:219) ~[drill-java-exec-1.1.0.jar:1.1.0] ... 7 common frames omitted {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)