drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From arina-ielchiieva <...@git.apache.org>
Subject [GitHub] drill pull request #1083: DRILL-4185: UNION ALL involving empty directory on...
Date Wed, 10 Jan 2018 15:52:27 GMT
Github user arina-ielchiieva commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1083#discussion_r160717181
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java
---
    @@ -250,20 +250,12 @@ private boolean metaDataFileExists(FileSystem fs, FileStatus dir)
throws IOExcep
         }
     
         boolean isDirReadable(DrillFileSystem fs, FileStatus dir) {
    -      Path p = new Path(dir.getPath(), ParquetFileWriter.PARQUET_METADATA_FILE);
           try {
    -        if (fs.exists(p)) {
    -          return true;
    -        } else {
    -
    -          if (metaDataFileExists(fs, dir)) {
    -            return true;
    -          }
    -          List<FileStatus> statuses = DrillFileSystemUtil.listFiles(fs, dir.getPath(),
false);
    -          return !statuses.isEmpty() && super.isFileReadable(fs, statuses.get(0));
    -        }
    +        // There should be at least one file, which is readable by Drill
    +        List<FileStatus> statuses = DrillFileSystemUtil.listFiles(fs, dir.getPath(),
false);
    +        return !statuses.isEmpty() && super.isFileReadable(fs, statuses.get(0));
    --- End diff --
    
    1. How we know that we have file in nested directory if filter checks with recursive flag
false?
    2. The `isReadable` method uses implicit assumption that if metadata file exists, there
is some data in folder. Plus comment in this file also  points out that the same logic is
used in `isDirReadable`. Since you are removing this logic from `isDirReadable` please make
sure the logic is synced between two methods or at least comment is updated to reflect new
approach.



---

Mime
View raw message