flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timo Walther <twal...@apache.org>
Subject Re: Flink 1.2-SNAPSHOT fails to initialize keyed state backend
Date Tue, 27 Sep 2016 15:04:21 GMT
Hi Seth,

the 1.2-SNAPSHOT is very fragile at the moment because of multiple big 
changes for dynamic scaling.
Maybe Stefan (in CC) has an idea what is happening with the keyed state 
backend here?

Timo


Am 27/09/16 um 16:14 schrieb swiesman:
> Hi all,
>
> I am working on an analytics project and am developing against flink
> 1.2-SNAPSHOT. The pipeline that I have built works; ie I can ingest data,
> perform operations, and output the expected result. I can also see
> checkpoints being written to RocksDB using Amazon S3 as the state backend.
> Whenever the application runs for more than approximately 5 - 10 minutes I
> get an RuntimeException: Could not initialize keyed state backed caused by a
> class not found exception, full stack trace posted below. This confuses me
> because for the program to have run and output data for the first 5 - 10
> minutes as it does that class was certainly loaded successfully in the past.
> As well, I am submitting a single fat jar to flink that I am certain
> contains the class in question.
>
> Thank you in advance for any assistance
>
> Seth Wiesman
>
> java.lang.RuntimeException: Could not initialize keyed state backend.
> 	at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.open(AbstractStreamOperator.java:148)
> 	at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:91)
> 	at
> org.apache.flink.streaming.api.operators.StreamFlatMap.open(StreamFlatMap.java:41)
> 	at
> org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:407)
> 	at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:283)
> 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:590)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException:
> com.mediamath.reporting.lib.streaming.UpsertTrigger
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> 	at java.lang.Class.forName0(Native Method)
> 	at java.lang.Class.forName(Class.java:348)
> 	at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:628)
> 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620)
> 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
> 	at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1486)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1336)
> 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
> 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
> 	at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
> 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
> 	at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:503)
> 	at
> org.apache.flink.api.common.state.StateDescriptor.readObject(StateDescriptor.java:311)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
> 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
> 	at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
> 	at
> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend$RocksDBRestoreOperation.restoreKVStateMetaData(RocksDBKeyedStateBackend.java:653)
> 	at
> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend$RocksDBRestoreOperation.restoreKeyGroupsInStateHandle(RocksDBKeyedStateBackend.java:626)
> 	at
> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend$RocksDBRestoreOperation.doRestore(RocksDBKeyedStateBackend.java:609)
> 	at
> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.<init>(RocksDBKeyedStateBackend.java:207)
> 	at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.restoreKeyedStateBackend(RocksDBStateBackend.java:269)
> 	at
> org.apache.flink.streaming.runtime.tasks.StreamTask.createKeyedStateBackend(StreamTask.java:778)
> 	at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.open(AbstractStreamOperator.java:142)
> 	... 6 more
>
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-2-SNAPSHOT-fails-to-initialize-keyed-state-backend-tp9208.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.


-- 
Freundliche Grüße / Kind Regards

Timo Walther

Follow me: @twalthr
https://www.linkedin.com/in/twalthr


Mime
View raw message