drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5783) Make code generation in the TopN operator more modular and test it
Date Wed, 18 Oct 2017 22:24:04 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16210173#comment-16210173
] 

ASF GitHub Bot commented on DRILL-5783:
---------------------------------------

Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/984#discussion_r145505747
  
    --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestCorruptParquetDateCorrection.java
---
    @@ -76,43 +78,50 @@
       //        - detecting corrupt values must be deferred to actual data page reading
       //    - one from 1.4, where there is a proper created-by, but the corruption is present
       private static final String MIXED_CORRUPTED_AND_CORRECT_DATES_PATH =
    -      "[WORKING_PATH]/src/test/resources/parquet/4203_corrupt_dates/mixed_drill_versions";
    +      "/parquet/4203_corrupt_dates/mixed_drill_versions";
       // partitioned with 1.2.0, no certain metadata that these were written with Drill
       // the value will be checked to see that they look corrupt and they will be corrected
       // by default. Users can use the format plugin option autoCorrectCorruptDates to disable
       // this behavior if they have foreign parquet files with valid rare date values that
are
       // in the similar range as Drill's corrupt values
    +  private static final String PARTITIONED_1_2_FOLDER = "partitioned_with_corruption_4203_1_2";
       private static final String CORRUPTED_PARTITIONED_DATES_1_2_PATH =
    -      "[WORKING_PATH]/src/test/resources/parquet/4203_corrupt_dates/partitioned_with_corruption_4203_1_2";
    +      Paths.get("/parquet/4203_corrupt_dates", PARTITIONED_1_2_FOLDER).toString();
       // partitioned with 1.4.0, no certain metadata regarding the date corruption status.
       // The same detection approach of the corrupt date values as for the files partitioned
with 1.2.0
    +  private static final String PARTITIONED_1_4_FOLDER = "partitioned_with_corruption_4203";
       private static final String CORRUPTED_PARTITIONED_DATES_1_4_0_PATH =
    -      "[WORKING_PATH]/src/test/resources/parquet/4203_corrupt_dates/partitioned_with_corruption_4203";
    +      Paths.get("/parquet/4203_corrupt_dates", PARTITIONED_1_4_FOLDER).toString();
       private static final String PARQUET_DATE_FILE_WITH_NULL_FILLED_COLS =
    -      "[WORKING_PATH]/src/test/resources/parquet/4203_corrupt_dates/null_date_cols_with_corruption_4203.parquet";
    +      "/parquet/4203_corrupt_dates/null_date_cols_with_corruption_4203.parquet";
    +  private static final String PARTITIONED_1_9_FOLDER = "1_9_0_partitioned_no_corruption";
       private static final String CORRECT_PARTITIONED_DATES_1_9_PATH =
    -      "[WORKING_PATH]/src/test/resources/parquet/4203_corrupt_dates/1_9_0_partitioned_no_corruption";
    +      Paths.get("/parquet/4203_corrupt_dates", PARTITIONED_1_9_FOLDER).toString();
       private static final String VARCHAR_PARTITIONED =
    -      "[WORKING_PATH]/src/test/resources/parquet/4203_corrupt_dates/fewtypes_varcharpartition";
    +      "/parquet/4203_corrupt_dates/fewtypes_varcharpartition";
       private static final String DATE_PARTITIONED =
    -      "[WORKING_PATH]/src/test/resources/parquet/4203_corrupt_dates/fewtypes_datepartition";
    +      "/parquet/4203_corrupt_dates/fewtypes_datepartition";
       private static final String EXCEPTION_WHILE_PARSING_CREATED_BY_META =
    -      "[WORKING_PATH]/src/test/resources/parquet/4203_corrupt_dates/hive1dot2_fewtypes_null";
    +      "/parquet/4203_corrupt_dates/hive1dot2_fewtypes_null";
       private static final String CORRECT_DATES_1_6_0_PATH =
    -      "[WORKING_PATH]/src/test/resources/parquet/4203_corrupt_dates/correct_dates_and_old_drill_parquet_writer.parquet";
    -  private static final String PARTITIONED_1_2_FOLDER = "partitioned_with_corruption_4203_1_2";
    +      "/parquet/4203_corrupt_dates/correct_dates_and_old_drill_parquet_writer.parquet";
       private static final String MIXED_CORRUPTED_AND_CORRECT_PARTITIONED_FOLDER = "mixed_partitioned";
     
    -
       @BeforeClass
       public static void initFs() throws Exception {
    +    dirTestWatcher.copyResourceToRoot("/parquet/4203_corrupt_dates");
    --- End diff --
    
    Each time I see this I have to remember that this is an absolute path relative to some
unstated root. That is, although it looks absolute, it is actually relative...


> Make code generation in the TopN operator more modular and test it
> ------------------------------------------------------------------
>
>                 Key: DRILL-5783
>                 URL: https://issues.apache.org/jira/browse/DRILL-5783
>             Project: Apache Drill
>          Issue Type: Improvement
>            Reporter: Timothy Farkas
>            Assignee: Timothy Farkas
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message