tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Turner Eagles (Jira)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-4298) ShuffleHandler is not source compatible with Hadoop 3.3.0
Date Fri, 25 Jun 2021 19:25:00 GMT

    [ https://issues.apache.org/jira/browse/TEZ-4298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369677#comment-17369677
] 

Jonathan Turner Eagles commented on TEZ-4298:
---------------------------------------------

[~tasanuma], I have pushed this change to branch-0.9. There is nothing specific to hadoop
3.3 about this change and it will help support both lines as it will be easier to back port
other changes to branch-0.9 without conflict.

As far as running tez 0.9 releases on hadoop 3.3, that is problematic. There are a number
conflicting dependencies with hadoop 3.3 that prevent hadoop 3.3 and tez 0.9 being in the
same classpath (jetty, protobuf) during runtime. Tez 0.9 and 0.10 are nearly identical with
the most significant change being hadoop 3.x compatibility. 

> ShuffleHandler is not source compatible with Hadoop 3.3.0
> ---------------------------------------------------------
>
>                 Key: TEZ-4298
>                 URL: https://issues.apache.org/jira/browse/TEZ-4298
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Jonathan Turner Eagles
>            Assignee: Jonathan Turner Eagles
>            Priority: Major
>             Fix For: 0.9.3, 0.10.2
>
>         Attachments: TEZ-4298.001.patch, TEZ-4298.001.patch, TEZ-4298.002.patch
>
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> {noformat}
> mvn clean package -DskipTests -pl '!tez-ui' -Dhadoop.version=3.3.0
> [ERROR] COMPILATION ERROR : 
> [INFO] -------------------------------------------------------------
> [ERROR] tez/tez-plugins/tez-aux-services/src/main/java/org/apache/tez/auxservices/ShuffleHandler.java:[764,45]
incompatible types: com.google.protobuf.ByteString cannot be converted to org.apache.hadoop.thirdparty.protobuf.ByteString
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message