flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Help using HBase with Flink 1.1.4
Date Tue, 17 Jan 2017 01:31:25 GMT
Logged FLINK-5517 for upgrading hbase version to 1.3.0

On Mon, Jan 16, 2017 at 5:26 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> hbase uses Guava 12.0.1 and Flink uses 18.0 where Stopwatch.<init>()V is
> no longer accessible.
> HBASE-14963 removes the use of Stopwatch at this location.
>
> hbase 1.3.0 RC has passed voting period.
>
> Please use 1.3.0 where you wouldn't see the IllegalAccessError
>
> On Mon, Jan 16, 2017 at 4:50 PM, Giuliano Caliari <
> giuliano.caliari@gmail.com> wrote:
>
>> Hello,
>>
>> I'm trying to use HBase on one of my stream transformations and I'm
>> running into the Guava/Stopwatch dependency problem
>>
>> java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V
from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
>>
>>
>> Reading on the problem it seems that there is a way to avoid it using
>> shading:
>> https://ci.apache.org/projects/flink/flink-docs-release-1.1/
>> setup/building.html#dependency-shading
>>
>> But I can't get it to work.
>> I followed the documented steps and it builds but when I try to run the
>> newly built version it fails when trying to connect to the Resource Manager:
>>
>> 2017-01-17 00:42:05,872 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>>                   - Using values:
>> 2017-01-17 00:42:05,872 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>>                   - TaskManager count = 4
>> 2017-01-17 00:42:05,873 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>>                   - JobManager memory = 1024
>> 2017-01-17 00:42:05,873 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>>                   - TaskManager memory = 32768
>> 2017-01-17 00:42:05,892 INFO  org.apache.hadoop.yarn.client.RMProxy
>>                     - Connecting to ResourceManager at /0.0.0.0:8032
>> 2017-01-17 00:42:07,023 INFO  org.apache.hadoop.ipc.Client
>>                    - Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>> Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
>> sleepTime=1000 MILLISECONDS)
>> 2017-01-17 00:42:08,024 INFO  org.apache.hadoop.ipc.Client
>>                    - Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>> Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
>> sleepTime=1000 MILLISECONDS)
>>
>>
>> I'm currently building version 1.1.4 of Flink based on the github repo.
>> Building it without shading (not executing `mvn clean install` on the
>> flink-dist sub-project) works fine until I try to use HBase, at which point
>> I get the Stopwatch exception.
>>
>> Has anyone been able to solve this?
>>
>> Thanks you,
>>
>> Giuliano Caliari
>> --
>> --
>> Giuliano Caliari (+55 11 984898464 <+55%2011%2098489-8464>)
>> <http://www.facebook.com/giuliano.caliari>+Google
>> <https://plus.google.com/u/0/104857507547056767808/posts>
>> Twitter <https://twitter.com/gcaliari>
>>
>> Master Software Engineer by Escola Politécnica da USP
>> Bachelor in Computer Science by Instituto de Matemática e Estatística da
>> USP
>>
>>
>

Mime
View raw message