flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: How to avoid breaking states when upgrading Flink job?
Date Thu, 30 Jun 2016 11:55:05 GMT
Hi Josh,
I think in your case the problem is that Scala might choose different names
for synthetic/generated classes. This will trip up the code that is trying
to restore from a snapshot that was done with an earlier version of the
code where classes where named differently.

I'm afraid I don't know how to solve this one right now, except by
switching to Java.

Cheers,
Aljoscha

On Thu, 30 Jun 2016 at 13:38 Maximilian Michels <mxm@apache.org> wrote:

> Hi Josh,
>
> You have to assign UIDs to all operators to change the topology. Plus,
> you have to add dummy operators for all UIDs which you removed; this
> is a limitation currently because Flink will attempt to find all UIDs
> of the old job.
>
> Cheers,
> Max
>
> On Wed, Jun 29, 2016 at 9:00 PM, Josh <jofo90@gmail.com> wrote:
> > Hi all,
> > Is there any information out there on how to avoid breaking saved
> > states/savepoints when making changes to a Flink job and redeploying it?
> >
> > I want to know how to avoid exceptions like this:
> >
> > java.lang.RuntimeException: Failed to deserialize state handle and setup
> > initial operator state.
> >       at org.apache.flink.runtime.taskmanager.Task.run(Task.java:551)
> >       at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.lang.ClassNotFoundException:
> > com.me.flink.MyJob$$anon$1$$anon$7$$anon$4
> >
> >
> > The best information I could find in the docs is here:
> >
> >
> https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/savepoints.html
> >
> >
> > Having made the suggested changes to my job (i.e. giving a uid to every
> > stateful sink and map function), what changes to the job/topology are
> then
> > allowed/not allowed?
> >
> >
> > If I'm 'naming' my states by providing uids, why does Flink need to look
> for
> > a specific class, like com.me.flink.MyJob$$anon$1$$anon$7$$anon$4 ?
> >
> >
> > Thanks for any advice,
> >
> > Josh
>

Mime
View raw message