flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Flink 1.2-SNAPSHOT fails to initialize keyed state backend
Date Wed, 28 Sep 2016 13:23:30 GMT
Hi!

This was a temporary regression on the snapshot that has been fixed a few
days ago. It should be in the snapshot repositories by now.

Can you check if the problem persists if you force an update of your
snapshots dependencies?

Greetings,
Stephan



On Tue, Sep 27, 2016 at 5:04 PM, Timo Walther <twalthr@apache.org> wrote:

> Hi Seth,
>
> the 1.2-SNAPSHOT is very fragile at the moment because of multiple big
> changes for dynamic scaling.
> Maybe Stefan (in CC) has an idea what is happening with the keyed state
> backend here?
>
> Timo
>
>
> Am 27/09/16 um 16:14 schrieb swiesman:
>
> Hi all,
>>
>> I am working on an analytics project and am developing against flink
>> 1.2-SNAPSHOT. The pipeline that I have built works; ie I can ingest data,
>> perform operations, and output the expected result. I can also see
>> checkpoints being written to RocksDB using Amazon S3 as the state backend.
>> Whenever the application runs for more than approximately 5 - 10 minutes I
>> get an RuntimeException: Could not initialize keyed state backed caused
>> by a
>> class not found exception, full stack trace posted below. This confuses me
>> because for the program to have run and output data for the first 5 - 10
>> minutes as it does that class was certainly loaded successfully in the
>> past.
>> As well, I am submitting a single fat jar to flink that I am certain
>> contains the class in question.
>>
>> Thank you in advance for any assistance
>>
>> Seth Wiesman
>>
>> java.lang.RuntimeException: Could not initialize keyed state backend.
>>         at
>> org.apache.flink.streaming.api.operators.AbstractStreamOpera
>> tor.open(AbstractStreamOperator.java:148)
>>         at
>> org.apache.flink.streaming.api.operators.AbstractUdfStreamOp
>> erator.open(AbstractUdfStreamOperator.java:91)
>>         at
>> org.apache.flink.streaming.api.operators.StreamFlatMap.open(
>> StreamFlatMap.java:41)
>>         at
>> org.apache.flink.streaming.runtime.tasks.StreamTask.openAllO
>> perators(StreamTask.java:407)
>>         at
>> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(
>> StreamTask.java:283)
>>         at org.apache.flink.runtime.taskmanager.Task.run(Task.java:590)
>>         at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.ClassNotFoundException:
>> com.mediamath.reporting.lib.streaming.UpsertTrigger
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>         at java.lang.Class.forName0(Native Method)
>>         at java.lang.Class.forName(Class.java:348)
>>         at java.io.ObjectInputStream.resolveClass(ObjectInputStream.
>> java:628)
>>         at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream
>> .java:1620)
>>         at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.
>> java:1521)
>>         at java.io.ObjectInputStream.readClass(ObjectInputStream.java:
>> 1486)
>>         at java.io.ObjectInputStream.readObject0(ObjectInputStream.java
>> :1336)
>>         at java.io.ObjectInputStream.defaultReadFields(ObjectInputStrea
>> m.java:2018)
>>         at java.io.ObjectInputStream.readSerialData(ObjectInputStream.
>> java:1942)
>>         at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
>>         at java.io.ObjectInputStream.readObject0(ObjectInputStream.java
>> :1353)
>>         at java.io.ObjectInputStream.defaultReadFields(ObjectInputStrea
>> m.java:2018)
>>         at java.io.ObjectInputStream.defaultReadObject(ObjectInputStrea
>> m.java:503)
>>         at
>> org.apache.flink.api.common.state.StateDescriptor.readObject
>> (StateDescriptor.java:311)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:62)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:498)
>>         at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass
>> .java:1058)
>>         at java.io.ObjectInputStream.readSerialData(ObjectInputStream.
>> java:1909)
>>         at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
>>         at java.io.ObjectInputStream.readObject0(ObjectInputStream.java
>> :1353)
>>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:
>> 373)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBa
>> ckend$RocksDBRestoreOperation.restoreKVStateMetaData(RocksDB
>> KeyedStateBackend.java:653)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBa
>> ckend$RocksDBRestoreOperation.restoreKeyGroupsInStateHandle(
>> RocksDBKeyedStateBackend.java:626)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBa
>> ckend$RocksDBRestoreOperation.doRestore(RocksDBKeyedStateBac
>> kend.java:609)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBa
>> ckend.<init>(RocksDBKeyedStateBackend.java:207)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>> .restoreKeyedStateBackend(RocksDBStateBackend.java:269)
>>         at
>> org.apache.flink.streaming.runtime.tasks.StreamTask.createKe
>> yedStateBackend(StreamTask.java:778)
>>         at
>> org.apache.flink.streaming.api.operators.AbstractStreamOpera
>> tor.open(AbstractStreamOperator.java:142)
>>         ... 6 more
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-flink-user-maili
>> ng-list-archive.2336050.n4.nabble.com/Flink-1-2-SNAPSHOT-
>> fails-to-initialize-keyed-state-backend-tp9208.html
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive at Nabble.com.
>>
>
>
> --
> Freundliche Grüße / Kind Regards
>
> Timo Walther
>
> Follow me: @twalthr
> https://www.linkedin.com/in/twalthr
>
>

Mime
View raw message