drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vdiravka <...@git.apache.org>
Subject [GitHub] drill pull request #877: DRILL-5660: Drill 1.10 queries fail due to Parquet ...
Date Tue, 15 Aug 2017 12:20:22 GMT
Github user vdiravka commented on a diff in the pull request:

    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
    @@ -633,13 +645,22 @@ public long getRowCount() {
        * @throws UserException when the updated selection is empty, this happens if the user
selects an empty folder.
       private FileSelection
    -  initFromMetadataCache(FileSelection selection, Path metaFilePath) throws IOException
    +  expandSelectionFromMetadataCache(FileSelection selection, Path metaFilePath) throws
IOException {
         // get the metadata for the root directory by reading the metadata file
         // parquetTableMetadata contains the metadata for all files in the selection root
folder, but we need to make sure
         // we only select the files that are part of selection (by setting fileSet appropriately)
    +    MetadataContext metaContext = selection.getMetaContext();
    +    // make sure that a metadata context is created since we are going to use metadata
    +    if (metaContext == null) {
    +      metaContext = new MetadataContext();
    +      selection.setMetaContext(metaContext);
    +    }
         // get (and set internal field) the metadata for the directory by reading the metadata
    -    this.parquetTableMetadata = Metadata.readBlockMeta(fs, metaFilePath.toString(), selection.getMetaContext(),
    +    parquetTableMetadata = Metadata.readBlockMeta(fs, metaFilePath, metaContext, formatConfig);
    +    if (ignoreExpandingSelection(parquetTableMetadata, selection)) {
    +      return selection;
    +    }
         if (formatConfig.autoCorrectCorruptDates) {
    --- End diff --

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message