drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4980) Upgrading of the approach of parquet date correctness status detection
Date Tue, 22 Nov 2016 10:51:58 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15686374#comment-15686374
] 

ASF GitHub Bot commented on DRILL-4980:
---------------------------------------

Github user vdiravka commented on a diff in the pull request:

    https://github.com/apache/drill/pull/644#discussion_r89088416
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetReaderUtility.java
---
    @@ -189,10 +195,16 @@ public static DateCorruptionStatus detectCorruptDates(ParquetMetadata
footer,
     
         String createdBy = footer.getFileMetaData().getCreatedBy();
         String drillVersion = footer.getFileMetaData().getKeyValueMetaData().get(ParquetRecordWriter.DRILL_VERSION_PROPERTY);
    -    String isDateCorrect = footer.getFileMetaData().getKeyValueMetaData().get(ParquetRecordWriter.IS_DATE_CORRECT_PROPERTY);
    +    String stringWriterVersion = footer.getFileMetaData().getKeyValueMetaData().get(ParquetRecordWriter.WRITER_VERSION_PROPERTY);
    +    // This flag can be present in parquet files which were generated with 1.9.0-SNAPSHOT
drill version.
    +    // If this flag is present it means that the version of the drill parquet writer
is 2
    +    final String isDateCorrectFlag = "is.date.correct";
    +    String isDateCorrect = footer.getFileMetaData().getKeyValueMetaData().get(isDateCorrectFlag);
         if (drillVersion != null) {
    -      return Boolean.valueOf(isDateCorrect) ? DateCorruptionStatus.META_SHOWS_NO_CORRUPTION
    -          : DateCorruptionStatus.META_SHOWS_CORRUPTION;
    +      int writerVersion = (stringWriterVersion != null) ? Integer.parseInt(stringWriterVersion)
    --- End diff --
    
    Agree. It makes sense. 
    I even found one redundant check in the return statement.
    Changes in a new commit.


> Upgrading of the approach of parquet date correctness status detection
> ----------------------------------------------------------------------
>
>                 Key: DRILL-4980
>                 URL: https://issues.apache.org/jira/browse/DRILL-4980
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - Parquet
>    Affects Versions: 1.8.0
>            Reporter: Vitalii Diravka
>            Assignee: Parth Chandra
>             Fix For: Future
>
>
> This jira is an addition for the [DRILL-4203|https://issues.apache.org/jira/browse/DRILL-4203].
> The date correctness label for the new generated parquet files should be upgraded. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message