apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raja.Aravapalli <Raja.Aravapa...@target.com>
Subject Re: Application restarts
Date Tue, 21 Jun 2016 20:56:29 GMT

Thanks Thomas, application is running fine then. Was just curious to know about the errors.


Regards,
Raja.

From: Thomas Weise <thomas.weise@gmail.com<mailto:thomas.weise@gmail.com>>
Reply-To: "users@apex.apache.org<mailto:users@apex.apache.org>" <users@apex.apache.org<mailto:users@apex.apache.org>>
Date: Tuesday, June 21, 2016 at 2:32 PM
To: "users@apex.apache.org<mailto:users@apex.apache.org>" <users@apex.apache.org<mailto:users@apex.apache.org>>
Subject: Re: Application restarts

This looks like the check of the last known container from the previous attempt when the AM
recovers from saved state. You can ignore this message if the application comes back up and
resumes processing. There is no data loss, the application resumes from checkpointed state.

On Tue, Jun 21, 2016 at 8:02 AM, Raja.Aravapalli <Raja.Aravapalli@target.com<mailto:Raja.Aravapalli@target.com>>
wrote:

Thanks Priyanka, this information helps.

Also, when I do restarts, I am also seeing below ERROR Exception in logs… Do we know what
they mean, and would that cause any data loss !!



2016-06-21 09:48:18,995 ERROR stram.StreamingAppMasterService (StreamingAppMasterService.java:onGetContainerStatusError(1164))
- Failed to query the status of container_e3_1463086279244_457551_01_1114442
org.apache.hadoop.security.token.SecretManager$InvalidToken: No NMToken sent for *******some_hostname



Thanks.


Regards,
Raja.


From: Priyanka Gugale <priyanka@datatorrent.com<mailto:priyanka@datatorrent.com>>
Reply-To: "users@apex.apache.org<mailto:users@apex.apache.org>" <users@apex.apache.org<mailto:users@apex.apache.org>>
Date: Tuesday, June 21, 2016 at 12:58 AM

To: "users@apex.apache.org<mailto:users@apex.apache.org>" <users@apex.apache.org<mailto:users@apex.apache.org>>
Subject: Re: Application restarts

Can you try @BindMap, check kyro README for how to use it: https://github.com/EsotericSoftware/kryo
Looks like it allows us to use different sterilizers for Map key and value fields.

-Priyanka

On Tue, Jun 21, 2016 at 10:59 AM, Raja.Aravapalli <Raja.Aravapalli@target.com<mailto:Raja.Aravapalli@target.com>>
wrote:

Hi Ram,

I have a variable like below,

private TreeMap<DateTime, ArrayList<String>> varMap;


During restarts, the DateTime key in the map is failing with serialization exceptions…

Can you please help me now….  How to annotate this variable to only take DateTime key for
JodaDateTime serilization  ?

Thanks a lot.


Regards,
Raja.

From: "Raja.Aravapalli" <Raja.Aravapalli@target.com<mailto:Raja.Aravapalli@target.com>>
Reply-To: "users@apex.apache.org<mailto:users@apex.apache.org>" <users@apex.apache.org<mailto:users@apex.apache.org>>
Date: Monday, June 20, 2016 at 6:48 PM

To: "users@apex.apache.org<mailto:users@apex.apache.org>" <users@apex.apache.org<mailto:users@apex.apache.org>>
Subject: Re: Application restarts


Thanks Ram.

I will test and let you know.


Regards,
Raja.

From: Munagala Ramanath <ram@datatorrent.com<mailto:ram@datatorrent.com>>
Reply-To: "users@apex.apache.org<mailto:users@apex.apache.org>" <users@apex.apache.org<mailto:users@apex.apache.org>>
Date: Monday, June 20, 2016 at 5:50 PM
To: "users@apex.apache.org<mailto:users@apex.apache.org>" <users@apex.apache.org<mailto:users@apex.apache.org>>
Subject: Re: Application restarts

If your field type is org.joda.time.DateTime and declared thus:

DateTime jodaDateTime;

How about just adding these imports:

import com.esotericsoftware.kryo.serializers.FieldSerializer.Bind;
import de.javakaffee.kryoserializers.jodatime.JodaDateTimeSerializer;

and adding this annotation just above the field declaration, like so:

@FieldSerializer.Bind(JodaDateTimeSerializer.class)
DateTime jodaDateTime;


Ram

On Mon, Jun 20, 2016 at 3:20 PM, Raja.Aravapalli <Raja.Aravapalli@target.com<mailto:Raja.Aravapalli@target.com>>
wrote:

Thanks a lot Ram for the response.

Yes, I am relaunching with application Id.

The usage related information in the documentation links are like bit heavy for the knowledge
I have… do we have a simple example on how to use?

By the way, I want to use JodaTime DateTime Serializer which are already available I believe.
Any help for using DateTime JodaDateTimeSerializer() ?

Thanks a lot.


Regards,
Raja.

From: Munagala Ramanath <ram@datatorrent.com<mailto:ram@datatorrent.com>>
Reply-To: "users@apex.apache.org<mailto:users@apex.apache.org>" <users@apex.apache.org<mailto:users@apex.apache.org>>
Date: Monday, June 20, 2016 at 4:32 PM
To: "users@apex.apache.org<mailto:users@apex.apache.org>" <users@apex.apache.org<mailto:users@apex.apache.org>>
Subject: Re: Application restarts

By "relaunches" I assume you mean you're launching with the previous application ID ?

When you do that, the platform attempts to restore state from the previously saved serialized
form.
To do that, it needs to firstcreate the object with a no-arg constructor and then populate
it.
If that constructor is absent, you get the exception.

Please see the section entitled "Application throwing following Kryo exception"
at http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception

Ram

On Mon, Jun 20, 2016 at 1:32 PM, Raja.Aravapalli <Raja.Aravapalli@target.com<mailto:Raja.Aravapalli@target.com>>
wrote:

Hi,

During application relaunches I am getting serialization exceptions like below, but not when
launching the application for the first time ….


Caused by: com.esotericsoftware.kryo.KryoException: Class cannot be created (missing no-arg
constructor): org.joda.time.chrono.ISOChronology
Serialization trace:
iChronology (org.joda.time.DateTime)



I am seeing these exception only when doing relaunches not for the first time launching!!
Relaunch can be any of the scenario like, KILLED or SHUTDOWN!!


Please share you thoughts….



Regards,

Raja.




Mime
View raw message