flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Giuliano Caliari <giuliano.cali...@gmail.com>
Subject Help using HBase with Flink 1.1.4
Date Tue, 17 Jan 2017 00:50:26 GMT
Hello,

I'm trying to use HBase on one of my stream transformations and I'm running
into the Guava/Stopwatch dependency problem

java.lang.IllegalAccessError: tried to access method
com.google.common.base.Stopwatch.<init>()V from class
org.apache.hadoop.hbase.zookeeper.MetaTableLocator


Reading on the problem it seems that there is a way to avoid it using
shading:
https://ci.apache.org/projects/flink/flink-docs-release-1.1/setup/building.html#dependency-shading

But I can't get it to work.
I followed the documented steps and it builds but when I try to run the
newly built version it fails when trying to connect to the Resource Manager:

2017-01-17 00:42:05,872 INFO  org.apache.flink.yarn.YarnClusterDescriptor
                - Using values:
2017-01-17 00:42:05,872 INFO  org.apache.flink.yarn.YarnClusterDescriptor
                - TaskManager count = 4
2017-01-17 00:42:05,873 INFO  org.apache.flink.yarn.YarnClusterDescriptor
                - JobManager memory = 1024
2017-01-17 00:42:05,873 INFO  org.apache.flink.yarn.YarnClusterDescriptor
                - TaskManager memory = 32768
2017-01-17 00:42:05,892 INFO  org.apache.hadoop.yarn.client.RMProxy
                - Connecting to ResourceManager at /0.0.0.0:8032
2017-01-17 00:42:07,023 INFO  org.apache.hadoop.ipc.Client
                 - Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
Already tried 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
MILLISECONDS)
2017-01-17 00:42:08,024 INFO  org.apache.hadoop.ipc.Client
                 - Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
Already tried 1 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
MILLISECONDS)


I'm currently building version 1.1.4 of Flink based on the github repo.
Building it without shading (not executing `mvn clean install` on the
flink-dist sub-project) works fine until I try to use HBase, at which point
I get the Stopwatch exception.

Has anyone been able to solve this?

Thanks you,

Giuliano Caliari
-- 
-- 
Giuliano Caliari (+55 11 984898464)
<http://www.facebook.com/giuliano.caliari>+Google
<https://plus.google.com/u/0/104857507547056767808/posts>
Twitter <https://twitter.com/gcaliari>

Master Software Engineer by Escola Politécnica da USP
Bachelor in Computer Science by Instituto de Matemática e Estatística da USP

Mime
View raw message