hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17908) Upgrade guava
Date Thu, 22 Jun 2017 21:51:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060061#comment-16060061
] 

stack commented on HBASE-17908:
-------------------------------

As part of this exercise, I purged all includes of guava, even transitives from hadoop. Here
are places where hadoop itself depends on guava so we'll have to let the transitives come
in. Seems tough to do any exclusion here if Hadoop Configuration depends on guava so I ended-up
putting back transitive include by hadoop-guava of whatever its guava is.

{code}
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.502 sec <<< FAILURE!
- in org.apache.hadoop.hbase.io.crypto.aes.TestAES
testAlternateRNG(org.apache.hadoop.hbase.io.crypto.aes.TestAES)  Time elapsed: 0.252 sec 
<<< ERROR!
java.lang.NoClassDefFoundError: com/google/common/base/Preconditions
  at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  at org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:328)
  at org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:341)
  at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:423)
  at org.apache.hadoop.hbase.io.crypto.aes.TestAES.testAlternateRNG(TestAES.java:88)
{code}

Let me post a patch that is severe in its guava eliminations and that has us using guava 22.0
(shaded).

One thing I think I have learned is that shading, relocations need to always be the same for
the case where a shaded lib itself depends on another shaded lib; e.g. protobuf utils depends
on guava. Currently we have our internal relocation for protobuf3 which adds the prefix org.apache.hadoop.hbase.shaded
but our thirdparty jar has guava offset by org.apache.hbase.thirdparty.shaded; I think these
need to be the same. Let me try moving protobuf to hbase-thirdparty project and bring in the
protobuf util which uses guava and get that working. Once that is done, I think we have a
pattern going forward for removing from the guava/netty/protobuf versions mash-up.

> Upgrade guava
> -------------
>
>                 Key: HBASE-17908
>                 URL: https://issues.apache.org/jira/browse/HBASE-17908
>             Project: HBase
>          Issue Type: Sub-task
>          Components: dependencies
>            Reporter: Balazs Meszaros
>            Assignee: stack
>            Priority: Critical
>             Fix For: 2.0.0
>
>
> Currently we are using guava 12.0.1, but the latest version is 21.0. Upgrading guava
is always a hassle because it is not always backward compatible with itself.
> Currently I think there are to approaches:
> 1. Upgrade guava to the newest version (21.0) and shade it.
> 2. Upgrade guava to a version which does not break or builds (15.0).
> If we can update it, some dependencies should be removed: commons-collections, commons-codec,
...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message