drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5978) Upgrade Hive libraries to 2.1.1 version.
Date Tue, 20 Feb 2018 23:07:03 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370717#comment-16370717
] 

ASF GitHub Bot commented on DRILL-5978:
---------------------------------------

Github user vdiravka commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1111#discussion_r169397714
  
    --- Diff: contrib/storage-hive/hive-exec-shade/pom.xml ---
    @@ -34,28 +34,47 @@
           <artifactId>hive-exec</artifactId>
           <scope>compile</scope>
           <exclusions>
    +        <!--Hive Calcite libraries are not required. When user submits query in Drill
via Hive plugin, the query
    +        is validated and planned via Drill Calcite. Hive Calcite can be used only to
setup Hive store for Drill unit
    +        testing, where a lot of Hive specific queries are performed. But Drill Calcite
and Avatica versions have
    +        conflicts with Hive old Calcite and Avatica versions. That's why Calcite cost
based optimizator
    +        (ConfVars.HIVE_CBO_ENABLED) is disabled for Drill Hive JUnit test cases. It can
be enabled again once Hive
    +        will leverage the newest Calcite version. To do that check whether Drill Calcite
and Avatica versions are
    +        suitable for hive-exec. If no, use Hive Calcite and Avatica versions.
    +        Note: Versions of Calcite libraries are controlled by "DependencyManagement"
block in Drill's
    +        root POM file now-->
             <exclusion>
    -          <artifactId>log4j</artifactId>
    -          <groupId>log4j</groupId>
    +          <groupId>org.apache.calcite</groupId>
    +          <artifactId>calcite-core</artifactId>
             </exclusion>
             <exclusion>
    -          <groupId>commons-codec</groupId>
    -          <artifactId>commons-codec</artifactId>
    +          <groupId>org.apache.calcite</groupId>
    +          <artifactId>calcite-avatica</artifactId>
    +        </exclusion>
    +        <exclusion>
    +          <groupId>org.apache.calcite</groupId>
    +          <artifactId>calcite-linq4j</artifactId>
             </exclusion>
             <exclusion>
    -          <artifactId>calcite-avatica</artifactId>
               <groupId>org.apache.calcite</groupId>
    +          <artifactId>calcite-druid</artifactId>
             </exclusion>
           </exclusions>
         </dependency>
    +    <!--Once newer hive-exec version leverages parquet-column 1.9.0, this dependency
can be deleted -->
    --- End diff --
    
    Done


> Upgrade Hive libraries to 2.1.1 version.
> ----------------------------------------
>
>                 Key: DRILL-5978
>                 URL: https://issues.apache.org/jira/browse/DRILL-5978
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - Hive
>    Affects Versions: 1.11.0
>            Reporter: Vitalii Diravka
>            Assignee: Vitalii Diravka
>            Priority: Major
>              Labels: doc-impacting
>             Fix For: 1.13.0
>
>
> Currently Drill uses [Hive version 1.2.1 libraries|https://github.com/apache/drill/blob/master/pom.xml#L53]
to perform queries on Hive. This version of library can be used for Hive1.x versions and Hive2.x
versions too, but some features of Hive2.x are broken (for example using of ORC transactional
tables). To fix that it will be good to update drill-hive library version to 2.1 or newer.

> Tasks which should be done:
> - resolving dependency conflicts;
> - investigating backward compatibility of newer drill-hive library with older Hive versions
(1.x);
> - updating drill-hive version for [MapR|https://github.com/apache/drill/blob/master/pom.xml#L1777]
profile too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message