orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From omalley <...@git.apache.org>
Subject [GitHub] orc pull request #306: ORC-363: Enable zstd for java writer/reader
Date Fri, 14 Sep 2018 20:27:20 GMT
Github user omalley commented on a diff in the pull request:

    https://github.com/apache/orc/pull/306#discussion_r217801734
  
    --- Diff: java/pom.xml ---
    @@ -68,8 +68,8 @@
         <test.tmp.dir>${project.build.directory}/testing-tmp</test.tmp.dir>
         <example.dir>${project.basedir}/../../examples</example.dir>
     
    -    <min.hadoop.version>2.2.0</min.hadoop.version>
    -    <hadoop.version>2.7.3</hadoop.version>
    +    <min.hadoop.version>2.9.0</min.hadoop.version>
    --- End diff --
    
    We shouldn't bump the minimum version of Hadoop to 2.9. Let's instead put it into the
shims. We'll need to add a method like createZstdCodec to HadoopShims. We'll also need a new
implementation for HadoopShimsPre2_9 that handles variable length blocks, but not zstd. You'll
also need a graceful message if they don't have the native libraries available.


---

Mime
View raw message