flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vishal Santoshi <vishal.santo...@gmail.com>
Subject Re: Few question about upgrade from 1.4 to 1.5 flink ( some very basic )
Date Mon, 25 Jun 2018 13:03:35 GMT
Thank you....

One addition

I do not see WM info on the UI  ( Attached )

Is this a know issue. The same pipe on our production has the WM ( In fact
never had an issue with  Watermarks not appearing ) . Am I missing
something ?

On Mon, Jun 25, 2018 at 4:15 AM, Fabian Hueske <fhueske@gmail.com> wrote:

> Hi Vishal,
>
> 1. I don't think a rolling update is possible. Flink 1.5.0 changed the
> process orchestration and how they communicate. IMO, the way to go is to
> start a Flink 1.5.0 cluster, take a savepoint on the running job, start
> from the savepoint on the new cluster and shut the old job down.
> 2. Savepoints should be compatible.
> 3. You can keep the slot configuration as before.
> 4. As I said before, mixing 1.5 and 1.4 processes does not work (or at
> least, it was not considered a design goal and nobody paid attention that
> it is possible).
>
> Best, Fabian
>
>
> 2018-06-23 13:38 GMT+02:00 Vishal Santoshi <vishal.santoshi@gmail.com>:
>
>>
>> 1.
>> Can or has any one  done  a rolling upgrade from 1.4 to 1.5 ?  I am not
>> sure we can. It seems that JM cannot recover jobs with this exception
>>
>> Caused by: java.io.InvalidClassException: org.apache.flink.runtime.jobgr
>> aph.tasks.CheckpointCoordinatorConfiguration; local class incompatible:
>> stream classdesc serialVersionUID = -647384516034982626, local class
>> serialVersionUID = 2
>>
>>
>>
>>
>> 2.
>> Does SP on 1.4, resume on 1.5 ( pretty basic but no harm asking ) ?
>>
>>
>>
>> 3.
>> https://ci.apache.org/projects/flink/flink-docs-release-1.5/
>> release-notes/flink-1.5.html#update-configuration-for-
>> reworked-job-deployment The taskmanager.numberOfTaskSlots: What would be
>> the desired setting in a stand alone ( non mesos/yarn ) cluster ?
>>
>>
>> 4. I suspend all jobs and establish 1.5 on the JM ( the TMs are still
>> running with 1.4 ) . JM refuse to start  with
>>
>> Jun 23 07:34:23 flink-ad21ac07.bf2.tumblr.net docker[3395]: 2018-06-23
>> 11:34:23 ERROR JobManager:116 - Failed to recover job
>> 454cd84a519f3b50e88bcb378d8a1330.
>>
>> Jun 23 07:34:23 flink-ad21ac07.bf2.tumblr.net docker[3395]:
>> java.lang.InstantiationError: org.apache.flink.runtime.blob.BlobKey
>>
>> Jun 23 07:34:23 flink-ad21ac07.bf2.tumblr.net docker[3395]: at
>> sun.reflect.GeneratedSerializationConstructorAccessor51.newInstance(Unknown
>> Source)
>>
>> Jun 23 07:34:23 flink-ad21ac07.bf2.tumblr.net docker[3395]: at
>> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>
>> Jun 23 07:34:23 flink-ad21ac07.bf2.tumblr.net docker[3395]: at
>> java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:1079)
>>
>> Jun
>> .....
>>
>>
>>
>> Any feedback would be highly appreciated...
>>
>>
>

Mime
View raw message