drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4185) UNION ALL involving empty directory on any side of union all results in Failed query
Date Thu, 11 Jan 2018 16:16:02 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322471#comment-16322471
] 

ASF GitHub Bot commented on DRILL-4185:
---------------------------------------

Github user vdiravka commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1083#discussion_r160997779
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java
---
    @@ -250,20 +250,12 @@ private boolean metaDataFileExists(FileSystem fs, FileStatus dir)
throws IOExcep
         }
     
         boolean isDirReadable(DrillFileSystem fs, FileStatus dir) {
    -      Path p = new Path(dir.getPath(), ParquetFileWriter.PARQUET_METADATA_FILE);
           try {
    -        if (fs.exists(p)) {
    -          return true;
    -        } else {
    -
    -          if (metaDataFileExists(fs, dir)) {
    -            return true;
    -          }
    -          List<FileStatus> statuses = DrillFileSystemUtil.listFiles(fs, dir.getPath(),
false);
    -          return !statuses.isEmpty() && super.isFileReadable(fs, statuses.get(0));
    -        }
    +        // There should be at least one file, which is readable by Drill
    +        List<FileStatus> statuses = DrillFileSystemUtil.listFiles(fs, dir.getPath(),
false);
    +        return !statuses.isEmpty() && super.isFileReadable(fs, statuses.get(0));
    --- End diff --
    
    I have reverted my changes here. SchemalessScan will be created if ParquetGroupScan has
empty entries and can't be used. The info that parquet metadata files are invalid will be
logged.


> UNION ALL involving empty directory on any side of union all results in Failed query
> ------------------------------------------------------------------------------------
>
>                 Key: DRILL-4185
>                 URL: https://issues.apache.org/jira/browse/DRILL-4185
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Relational Operators
>    Affects Versions: 1.4.0
>            Reporter: Khurram Faraaz
>            Assignee: Vitalii Diravka
>
> UNION ALL query that involves an empty directory on either side of UNION ALL operator
results in FAILED query. We should return the results for the non-empty side (input) of UNION
ALL.
> Note that empty_DIR is an empty directory, the directory exists, but it has no files
in it. 
> Drill 1.4 git.commit.id=b9068117
> 4 node cluster on CentOS
> {code}
> 0: jdbc:drill:schema=dfs.tmp> select columns[0] from empty_DIR UNION ALL select cast(columns[0]
as int) c1 from `testWindow.csv`;
> Error: VALIDATION ERROR: From line 1, column 24 to line 1, column 32: Table 'empty_DIR'
not found
> [Error Id: 5c024786-6703-4107-8a4a-16c96097be08 on centos-01.qa.lab:31010] (state=,code=0)
> 0: jdbc:drill:schema=dfs.tmp> select cast(columns[0] as int) c1 from `testWindow.csv`
UNION ALL select columns[0] from empty_DIR;
> Error: VALIDATION ERROR: From line 1, column 90 to line 1, column 98: Table 'empty_DIR'
not found
> [Error Id: 58c98bc4-99df-425c-aa07-c8c5faec4748 on centos-01.qa.lab:31010] (state=,code=0)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message