flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-6025) User code ClassLoader not used when KryoSerializer fallbacks to serialization for copying
Date Sat, 11 Mar 2017 11:00:06 GMT

     [ https://issues.apache.org/jira/browse/FLINK-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tzu-Li (Gordon) Tai updated FLINK-6025:
---------------------------------------
    Description: 
Reported in ML: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/AWS-exception-serialization-problem-td12063.html

This is caused by a known Kryo issue with its {{JavaSerializer}}: https://github.com/EsotericSoftware/kryo/pull/483.

It happens when a {{Throwable}} is to be copied by the {{KryoSerialzer}}. Since we use the
{{JavaSerializer}} for throwables, and {{JavaSerializer}} doesn't support copying, the {{KryoSerializer}}
fallbacks to use de-/serialization for the throwable. The problem is that on deserialization,
the classloader that the {{ObjectInputStream}} uses may be overriden, and doesn't specifically
uses Kryo's configured classloader (i.e., the user code class loader), and results in {{ClassNotFoundException}}.

Generally, this may happen if the user also registers to use the {{JavaSerializer}} for their
types.

To fix the problem for {{Throwable}} serializing in the {{KryoSerializer}}, we could either
consider registering our own fixed {{JavaSerializer}} for throwables, or wait for the Kryo
fix to be released (to be fixed in Kryo 4.0.1 release).

  was:
Reported in ML: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/AWS-exception-serialization-problem-td12063.html

This is caused by a known Kryo issue with its {{JavaSerializer}}: https://github.com/EsotericSoftware/kryo/pull/483.

It happens when a {{Throwable}} is to be copied by the {{KryoSerialzer}}. Since we use the
{{JavaSerializer}} for throwables, and {{JavaSerializer}} doesn't support copying, the {{KryoSerializer}}
fallbacks to use de-/serialization for the throwable. The problem is that on deserialization,
the classloader that the {{ObjectInputStream}} uses may be overriden, and doesn't specifically
uses Kryo's configured classloader (i.e., the user code class loader), and results in {{ClassNotFoundException}}.

Generally, this may happen if the user also registers to use the {{JavaSerializer}} for their
types.

To fix the problem for {{Throwable}} serializing in the {{KryoSerializer}}, we could either
consider registering our own {{JavaSerializer}} that fixes this problem, or wait for the Kryo
fix to be released (to be fixed in Kryo 4.0.1 release).


> User code ClassLoader not used when KryoSerializer fallbacks to serialization for copying
> -----------------------------------------------------------------------------------------
>
>                 Key: FLINK-6025
>                 URL: https://issues.apache.org/jira/browse/FLINK-6025
>             Project: Flink
>          Issue Type: Bug
>          Components: Type Serialization System
>            Reporter: Tzu-Li (Gordon) Tai
>
> Reported in ML: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/AWS-exception-serialization-problem-td12063.html
> This is caused by a known Kryo issue with its {{JavaSerializer}}: https://github.com/EsotericSoftware/kryo/pull/483.
> It happens when a {{Throwable}} is to be copied by the {{KryoSerialzer}}. Since we use
the {{JavaSerializer}} for throwables, and {{JavaSerializer}} doesn't support copying, the
{{KryoSerializer}} fallbacks to use de-/serialization for the throwable. The problem is that
on deserialization, the classloader that the {{ObjectInputStream}} uses may be overriden,
and doesn't specifically uses Kryo's configured classloader (i.e., the user code class loader),
and results in {{ClassNotFoundException}}.
> Generally, this may happen if the user also registers to use the {{JavaSerializer}} for
their types.
> To fix the problem for {{Throwable}} serializing in the {{KryoSerializer}}, we could
either consider registering our own fixed {{JavaSerializer}} for throwables, or wait for the
Kryo fix to be released (to be fixed in Kryo 4.0.1 release).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message